The AR Assisted Maintenance Tool is a wearable augmented reality system built to provide live guidance on maintenance tasks for NASA

In 2011, we designed a system consisting of a Vuzix HMD, camera, and a wrist-worn controller that allowed an operator to step through an augmented reality guided procedure.

6 Years later in 2017 we revisited the procedure with the HoloLens (1st gen) and were able to provide a completely different experience, dramatically improved. This would go on to be used aboard the International Space Station for maintenance of the T2 treadmill.

In 2021 we flew the T2AR (Treadmill 2 Augmented Reality) Procedure tool on the HoloLens aboard the International Space Station, the culmination of years of experimentation and refinement.

Making a Complex Task Simple

  • The space was feature mapped, allowing us to overlay augmentation atop the hardware.

  • The system was (for the time) compact and lightweight and self-contained.

  • A wrist-worn Android phone was used a gesture interface device and allowed the user to easily and intuitively move forwards and backwards through the procedure. The gestures allowed the user to mark steps as complete, skipped, or input responses to the procedure.

Revisiting the Problem

  • With the advent of the HoloLens we were excited to give the procedure a second look. The system was rewritten from the ground up and made to take advantage of the new platforms advanced features.

  • The improvements were dramatic, and we were able to provide accurate and rich 3D overlays with marked improvements in tester performance.

  • A wrist-worn Android phone was used a gesture interface device and allowed the user to easily and intuitively move forwards and backwards through the procedure. The gestures allowed the user to mark steps as complete, skipped, or input responses to the procedure.

Deploying to Space

  • Leveraging the new architecture, the application was deployed to the International Space Station in 2021 where it was used to perform maintenance on the treadmill

  • Instructions that would otherwise require a lengthy set of written instructions were instead rendered as intuitive animations, showing the crew precisely what actions to take.