Expansion of AR Smartglasses in Chile – KTI Ingeniería deploys Vuzix in Chile

CMPC is a multinational corporation that produces and markets wood, pulp, packaging products and household and personal use products. “We implemented the use of Vuzix Smart Glasses in our company because we cannot travel sporadically to the more than 40 countries where we sell our products.

The objective of implementing this cutting-edge technology is to have greater access to our clients and take immediate resolutions,” explained Pedro Silva, Cardboard Technical Assistance Supervisor of CMPC’s Technical Deputy Manager.

Statkraft is an international energy producer of hydropower, wind power, solar power and gas-fired power and supplier of district heating. “Our plant is 45 minutes away, and each specialist has to constantly be traveling to operations.

Benefits of Smart glasses

With the implementation of Vuzix Smart Glasses in our facilities, we were able to realize a reduction of trips and travel costs. Recently, we were able to remotely obtain the advice of an expert in Brazil thanks to these glasses,” stated Mirko Ojeda, Dispatch Coordinator at Statkraft Chile.

Read more at https://www.vuzix.eu/News




How Technology Is Transforming The Education Sector

With the advent of technological innovations across industries, the education sector turns out to be the most influenced. In-fact, technology has been playing a prominent role at the forefront of education ever since learning and development came into being, right from carving symbols and figures on walls of caves, to Gurukul education where the students were taught the use of the technology prevailing then, to using of artificial intelligence (AI) and virtual reality (VR).

Modern technology has completely reshaped the entire education system. The digitally empowered classrooms over the internet have made education available to each and everyone who wants to learn across the world, anytime, any subject and anywhere. There is no limit to the strength of the classroom. Unlike physical classrooms that are limited to a maximum of sixty students, any number of students can access the virtual classrooms. When it comes to learning, there is an unlimited amount of knowledge available at no cost to an economically viable price.

Interestingly, according to Wikipedia, YouTube has more than seventy lakhs of educational videos, and besides, there are a lot of other educational websites that have various knowledge-sharing pages provided by respected subject experts in every field.

The effect of technology on the students

Not long ago, education was pertained to the reading of books and listening to teachers which were boring to many students and tiresome to teachers. Some educational institutes tried to introduce activity-based education which of course motivated the students and increased the interest level to a certain extend, but the effect was not as expected.

The education using modern technology like Augmented Reality, Virtual Reality and Artificial Intelligence has made learning more collaborative and engaging. An article by Schindler et al., 2017  states that Technological application in education engages the student to involve in high-order thinking, develop communication and discussion, and reflect on the gist of the content. It also enhances digital competency. Another research established that the implementation of technology in the classroom has enhanced the motivation of the student to understand and accomplish the tasks (Mistler-Jackson & Songer, 2000).

Undoubtedly, technology increased the interest in learning by many folds and modern technology helped the student improve their critical thinking and analytical skills which is very much necessary to face any kind of challenge. It has not only helped the students to become successful but also to excel. This is not only for schools but also for higher and professional studies.

Traditional teaching versus Virtual teaching

Marc Prensky [Educational Author 2001] noticed that an average student spends less than five thousand hours reading in his entire life but more than ten thousand hours playing digital and online games. He also stated that the students of the present are no longer the ones our traditional education system was designed to teach.

Other researchers such as Pucel and Stertz [2005], Crowe [2004], Lu and Gordon [2009], have recognized that technological education method is needed than traditional education.  The National School Boards Association [2007] recognized technological proficiency as an essential learning tool of the twenty-first century. A remarkable statement of John Dewey “If we teach today’s students as we taught yesterday’s we rob them of tomorrow” (Agnello, White, & Fryer, 2006) sums up the importance of technology in the education system.

Challenges in implementing technology in the schools and collages

Jung pointed out the challenges, the traditional classroom teachers have to face due to rapid shift and use of technological methods in the classroom and about the expansion of the knowledge base available. Gressard and Loyd (1985) said that the attitude of teachers towards technology and technical gadgets is the main factor in implementing Information technology in the education system. They also pointed that not all teachers are keen on technological methods.

Another point is that they have to be trained to upgrade themselves to be in line with upcoming technology. Most of the teachers from GenX feel that it is hard to learn the complicated technology and blackboards are much simple. They believe classroom education involves emotional binding that helps in being good humans. The teachers of millennials though not new to technology are busy and lack time to upgrade themselves. The other barriers are lack of resources, limited or no access, low expertise, limited support and lack of time. Butler and Sellbom (2002) and Chizmar & Williams (2001) stress reliability and hardware compatibility and internet issues. Technology is growing at an exponentially faster phase which means a gadget bought today may become less useful or not at all in three months. Upgrading those needs money, time and expertise.

Many believe that technology in the education sector would make students more attached to machines and less socially able. We can see that the children of iGen spend more time with cell phones and other gadgets. Even a baby less than a year becomes calm and is more attracted to the cell phone display. Studies are going on as to how it would affect the brain but for now, artificially intelligent machines are in a major role.

The good part is kids get easily adapted to technology. Software programming is becoming the favorite subject for kids these days. They could learn, analyze and create a software program even before they turn into teens. This proves that the brain is slowly evolving from its present state to a higher state of understanding machine language. There are also a lot of social groups online related to every field where the students interact with others with the same mindset. Group discussions and knowledge-sharing blogs are increasing so it is not fully right to say that society is becoming less socially able. It is only that the medium of communication has taken a different platform.

A good learning technological media can be something that is easily accessible in any place. It should be easy to use by an average person with limited knowledge of computers. It should be highly interactive, fun and work on low bandwidth and should not take lots of time to load, and should be able to upgrade itself from time to time.

 




How augmented reality can advance nuclear operations

Utilities can invest a million dollars and more to train an individual operator, over 18 months and more. The goal is exceptional competence and procedural adherence, and a solid knowledge of plant design and theory. But even with the best training, there is no replacement for operators with hands-on experience, who possess a level of knowledge unobtainable in a classroom or training programme. Utilities need put this invaluable knowledge to work.

Simultaneously, energy companies need to upskill frontline veterans. Additional skillsets are frequently at technology’s cutting-edge and it is essential to provide ongoing and regular training to operators.

Technological advances are providing innovative ways to tap into the knowledge and skillsets of experienced workers and train new generations of operators on core procedures and new protocols. Specifically, advanced augmented reality (AR) systems that enable nuclear facility operators to keep fully up to speed on best practices, and empower them to access expert instruction on rarely used procedures when necessary.

The role of AR can be tied to a SMART approach to operation fundamentals:

  • Solid knowledge – Does what you’re doing deliver powerful, actionable knowledge to the individual, and empower them to make the right decision at the right time?
  • Monitor – Do you have the ability to monitor the facility effectively?
  • Act with a conservative bias – Are you installing technology for technology’s sake? Do you need it? Does it help?
  • Rigorous control – Are you considering all the risks that are involved, both the physical risks and the cyber risks? Are you taking proactive steps to control and manage these risks?
  • Teamwork – Have you considered more than just the obvious users? Account for technologically-advanced operators, as well as team members who may prefer pen and paper. Make the technology easy to use and indispensable for all individuals.

Capturing valuable organisational knowledge

Unique information — equipment-specific maintenance and repair procedures, system insights, insights into available organisational resources, even awareness of management style and company culture  is invaluable company-specific knowledge.

This unique knowledge set is supplemented by new and rapidly expanding SCADA/ industrial internet of things (IIoT) and related data, provided by a new generation of sensors. Sensors are being incorporated into legacy equipment, as well as being standard on new equipment.

Advanced AR systems can address the need for ‘Solid knowledge’ (above), capturing utility-specific knowledge and leveraging system data, turning it into procedures all frontline staff can use. The result is improved operational efficiency, increased safety, and consistent completion of complex tasks, which raises employee productivity and allows utilities to operate safely and profitably.

Deliver real-time access to authored content

With instruction provided through AR, operators can walk up to a piece of equipment and follow step-by-step instructions to get the job done using a phone or tablet, or a headset when hands-free operation is needed. Performing complex tasks consistently and efficiently is a critical goal.

The ability for experts to author content themselves is essential, especially when it comes to communicating procedures for infrequent maintenance tasks such as reactor head venting.

With this need in mind, AR systems such as Manifest from Taqtile are designed to be intuitive. Knowledge experts are able to create step-by-step digital work instructions overlaid on facility equipment, which other operators view via their device or headset. The author can also make available supporting materials assisting in the repair or maintenance process, such as photographs, videos, PDFs, or manuals and documents, delivered in a hands-free heads-up display when needed.

Real-time access to content, ranging from step-by-step video instruction walking frontline staff through complex procedures to detailed holograms, will increase AR’s value. And when needed, live remote assist from the facility’s operations centre, utility headquarters or equipment OEMs, can be made available.

To assist in auditing, capturing evidence — a picture or video showing the proper performance, or visuals demonstrating completion of a task — is critical. Capturing evidence also helps orchestrate operations across teams, where one operator may be located at a piece of equipment, another at an electrical panel and a third in the control room. Each has their specific task in the executing a complex procedure, and AR platforms can help guide each individual while keeping other team members apprised of status and progress in real time.

The ability to enable two-way video, real-time remote guidance, and on-site access to repositories of knowledge, helps remove ambiguity from the communication process and empowers workers to complete complex tasks at the machine.

Leverage the power of 5G

Nuclear facilities are considering adoption of 5G, driven by the edge-computing capabilities, support for remote operations and improved security and reliability. With 5G, the capabilities and value of AR applications increase, as 5G makes data more portable and AR systems faster, more secure and more resilient.

Maximising expertise within organisations is also a supported by AR and optimised by 5G. AR platforms can capture and disseminate knowledge from experts and distribute it to other staff, regardless of location, making employees more productive more quickly, with reduced classroom time.

The integration of AR with IIoT technologies will be greatly enhanced by 5G and the incorporation of sensors into legacy equipment. It will provide transparent information on demand, right at the location of frontline workers, enabling them to directly and securely access critical data. 5G also reduces latency, increases throughput and enables the deployment and connection of more sensors, providing more data for machine learning to help inform maintenance tasks.

Meanwhile advances in measures for device authentication, encryption of user traffic between devices and the network, and additional security protocols provide a new level of security required by utilities.

The Manifest AR solution

The Manifest platform is an end-to-end augmented work instruction platform that allows nuclear utilities to capture knowledge that enables interactive work instructions, and deliver ‘over-the-shoulder’ support when needed.

The platform functions like an advanced help centre for frontline workers. Benefits include increased staff efficiency and accuracy during complex operations and maintenance tasks.

Real-time interactions between remote experts and field operators have traditionally been limited to simple audio communication. Manifest changes that by fully leveraging the power of AR, sharing the complete experience of the onsite operator with the remote expert through enhanced AR communication on most popular AR-enabled devices – HoloLens, Magic Leap, iPad, Trimble XR10, and Android phones. For example, an expert working from a PC or tablet in a remote location can follow the operator through a first-person view of a valve line up verification in containment, seeing precisely where operators are looking with tracked eye movement. This capability mitigates the risk of mispositions and human error. By tracking the operator’s view, a remote expert is able to guide the operator with real-time instruction, including annotating directly in the operator’s environment, using directional arrows or highlights to guide the operator.

Help-centre-like interactions available via the Manifest platform between remote expert and frontline worker enhance the capabilities and efficiency of all staff:

  • Virtual ink – Experts can draw and place targets over the operator’s field of view, and incorporate spatial audio, producing sounds that emanate from a position within the operator’s environment for easier location finding.
  • Hologram sharing – In addition to the video feed delivered from the headset, operators can share holograms, including any Manifest user interface or 3D models in their view.
  • Wayfinding – Remote experts can render holographic directional arrows in the operator’s field of view to guide them within the onsite environment.
  • Eye tracking – By tracking the operator’s eye position and projecting a directional image to the expert’s desktop, the expert can follow what the operator is looking at, enabling more accurate remote assistance.

For example, Taqtile industrial customers have experienced remarkable improvements in trainee performance, with some companies decreasing training times by 700 percent. Simultaneously, it has helped increase accuracy, reducing operator errors to zero in some cases.

Manifest can also capture evidence and operational data as part of regular inspections and preventative maintenance, supporting regulatory requirements and meeting audit and related oversight needs.

From improving training performance to upskilling frontline workers on new technologies, AR will enable utilities to capture the knowledge of their most seasoned operators and share that knowledge in-real-time across the organisation.

 




Healthcare Industry to Benefit from Marriage of 5G and Smart Glasses

For several hospitals, disruption led to innovation. Vuzix Smart Glasses allowed medical instructors to stream their first-person view to off-site students. Meanwhile, caregivers could provide doctors with a real-time view of patients.

Even though much of the developed world has now been vaccinated, healthcare administrations are doubling-down on this technology.

And with lower latency, higher capacity, and increased bandwidth, expect 5G to trigger a hands-free revolution in the healthcare space.

5G networks offer lower latency, higher capacity, and increased bandwidth, meaning wearable displays can more seamlessly assist healthcare workers. As a result, more lives will be saved.

As providers like Verizon (Vuzix Smart Glasses support BlueJeans by Verizon) roll-out 5G networks across North America and beyond, there’s more incentive for hospitals to equip staff with wearable displays.

And the motivation isn’t purely financial. Smart Glasses have a track record of making workers in several industries more efficient.

Smart Glasses enable medical practitioners to get real-time monitoring of a patient without the need for a face-to-face consultation.

This ability has value that extends far beyond the current public health crisis. Most notably, it allows experts to remotely treat patients. World-class practitioners are no longer required to travel to save lives.

Ultimately, wearable displays make healthcare workers more effective.

It’s time to ditch the clipboard. Hospitals are about to get a much-needed productivity boost.

See Vuzix’s AREA member profile.




Masters of Pie Reinventing Intelligence Operations for the British Army

Reinventing Intelligence Operations for the British Army

Masters of Pie was proud to showcase their work on the Digital Bird Table solution for the British Army at DSEI in London. The Digital Bird Table solution complements Project Hydra, a project about collating multiple data feeds and processing this information so that Command Centres are able to make better and more timely decisions.

The Masters of Pie’s software (Radical) has been integrated directly into multiple data sources such as SAP HANA and ESRI ArcGIS. The Radical real-time collaboration server pulls the Digital Bird Table experience together, synchronising Hololens and desktop users. The time-lapse video highlights how the solution was polling live data and tracking assets on the move. Watch Video

XR Collaboration for CAD/PLM and BIM Data

The XR collaboration for manufacturing and engineering consisted of a live demonstration of the Radical software integration into the CAD software application to enable the seamless transition to a single user or collaborative VR experience without the need for exporting complex and sensitive data into a standalone application. The solution is ideal for remote assistance, remote diagnostics, design review, design to manufacturing handover, design to supplier engagements, and engineering training purposes. Watch Video

DSEI was a great opportunity to showcase the Radical’s software ability to deliver such contrasting use cases, integrate into a wide array of data sources, deliver multiple end-user experiences (Hololens (AR), VR, and desktop), and support a range of deployment options (cloud, on-premise and stand-alone). Radical is the future of collaboration delivering any data securely to any device.




PTC Vuforia Enterprise AR Solutions Suite Attains “Best in Class” Ranking

According to the report, PTC provides the leading AR platform for connected workers in the market today for three reasons.  First, PTC is the only vendor that offers a broad solution portfolio for different use cases and on different technical levels – deep code (Vuforia Engine™), low code (Vuforia Studio™), and no code (Vuforia Expert Capture™Vuforia Instruct™, and Vuforia Chalk™).  Second, PTC is the only vendor that strongly pushes the integration of AR with emerging technologies such as IoT, AI, SaaS, and spatial computing.  Third, PTC has a strong vision for future AR integrations, bringing its Vuforia portfolio elements closer together to offer tightly integrated solutions for holistic work procedures.

In addition, the PAC Radar analysis recognizes PTC as the first vendor to integrate its AR and IoT capabilities – Vuforia and its ThingWorx® IIoT platform – acknowledging PTC’s ability to harness the digital thread to create a closed loop between the digital and physical worlds and transform how products are engineered, manufactured, and serviced.

“No other vendor offers a broader solution portfolio for different use cases and on different technical levels” said Arnold Vogt, Head of Digital Innovation and IoT at PAC.  “PTC has a clear vision and has already started to bring its Vuforia portfolio elements closer together to offer tightly integrated solutions for holistic work procedures.”

“PTC is proud to be recognized as the leading AR platform for connected workers for the third consecutive year by the PAC Radar analysis,” said Michael Campbell, Executive Vice President and General Manager, Augmented Reality Products, PTC.  “The achievement emphasizes our commitment to offering the best-in-class augmented reality solutions that empower industrial enterprises and complete the digital thread.”

 




How AR is Benefitting Field Services

The pandemic completely altered the business world as we knew it, forcing organizations across all industries to quickly adapt to an entirely new way of operating. Companies were not prepared for the world to shut down and, as a result, had to digitally transform seemingly overnight. Now, nearly a year and a half later, organizations have evolved in this new normal, adopting fully digital operations and figuring out how to execute at the same standard in a remote world.

One such industry that experienced a drastic change in 2020 is field services. By relying on the right technology, field service companies are figuring out how to plan and control what they can while pivoting quickly to adapt to the things they can’t. Among the most valuable tools needed for field service companies to succeed in this landscape is augmented reality (AR).

Migrating to Remote Field Service Operations

Remote collaboration. Visual assistance. Contactless service. We are now all too familiar with these terms. Software solutions that provide remote visual assistance capabilities have been the most sought-after in the COVID era. One of the best and most widely adopted examples of AR in field service comes in the form of remote visual assistance software.

In short, remote assistance software is a cloud-based solution that provides a hands-on approach to problem-solving regardless of location. Enabled by AR, remote visual assistance technology creates a virtual environment where two parties unite on a real-time video call to overcome an issue as if they were working together in person, standing side by side.

By harnessing the power of AR features like merged reality and 3D annotation, remote visual assistance software creates a collaborative virtual environment where an expert can “reach out and touch” a piece of equipment that needs to be fixed or serviced. Let’s say, for example, the office printer isn’t printing. As a customer, you know the ink cartridges and paper trays are full, but the printer is still not responding.

With remote visual assistance technology, a remote expert can show you what fixes are required, whether replacing a cartridge, clearing a paper jam or tightening a nut or bolt. The expert’s hands are directly in the customer’s field of view, showing them precisely how to solve the problem.

Overcoming the Technician Talent Shortage

Remote visual assistance software enables companies offering field service support to optimize the time and effectiveness of their technicians. Like many industries, the field service is currently battling a significant talent shortage. In fact, according to 2020 research from the Service Council, nearly 57% of field service organizations are struggling to find enough talented technicians.

Senior technicians near retirement and looking to reduce their travel or fieldwork don’t have to leave the industry altogether. Instead, they can rely on remote visual assistance technology to deliver their expertise from a preferred location. A company’s field service organizations can extend the life of their valuable industry experts with remote visual assistance software by allowing them to work every day from a home or office rather than requiring them to travel on-site.

For field service technicians who aren’t preparing for retirement, remote visual assistance software provides the opportunity for these professionals to reimagine their roles. As organizations continue to utilize new technology, the amount of on-site support needed will likely continue to decrease. As a result, technicians will be able to focus more on customers – learning their needs, gaining their loyalty and boosting their satisfaction. With remote visual assistance software at work, field service technicians can transform their roles, adding more value and improving customer satisfaction.

Finally, remote visual assistance software is also an incredible tool to recruit a younger generation of technicians. It’s no surprise, younger technicians are more digitally savvy than previous generations and understand the value these tools bring to their positions. By offering remote visual assistance software, companies with a field service organization have a leg up in attracting new talent.

One thing is for sure: the field service industry isn’t going back to the old way of doing business. As more companies figure out how to transition to digital operations, remote visual assistance software will be essential for a successful transformation. While there will always be some expectation and value for in-person services, remote assistance technology will become the service delivery model of choice for customers and field service technicians alike.

 




Danfoss Climate Solutions using AR by Holo-Light for Product Training

When the stakes are high, the tools you use to communicate your point or value proposition will matter the most: Augmented reality has transformed how global engineering innovator Danfoss Climate Solutions has approached product presentations and trainings for its internal stakeholders and customers. The company uses the AR software AR 3S to visualize complex products in 3D to provide a captivating experience of learning and understanding.

Choosing the Right Augmented Reality Software and Partner~Keeping in line with the company’s DNA of “engineering tomorrow”, Danfoss Climate Solutions was looking to drive digital transformation by implementing augmented reality into its processes.

The company reached out to various AR suppliers, seeking high flexibility and a strong support culture. “We chose Holo-Light as our entryway to augmented reality engineering. It was easy to discuss use cases and solutions,” remarked Kabileshkumar Cheetancheri, Senior Project Manager at Danfoss.

Holo-Light’s software AR3S (Augmented Reality Engineering Space) is a new tool to improve and simplify workflows in planning, development, and presentation. It enables users to visualize, manipulate, and work collaboratively on 3D designs. “One part I was always interested in, is how we can add a new dimension to interacting with and explaining complex products – internally and externally for our customers,” said Cheetancheri. The product presentation and training use case emerged quickly as the AR software showed itself to be very easy and intuitive in use. With AR3S, Danfoss Climate Solutions was able to add real value to internal product demos with company stakeholders and help customers have a better understanding of its products.

 

Full original article on Holo-Light website

Find out more about Holo-Light on their AREA member profile 




Using Augmented Reality For Hands-On Training

What are the benefits?

Augmented reality provides many benefits for educating and training employees. Visual and hands-on learners may learn more from augmented reality than from a lecture or conference. They can practice as many times as needed without dangerous consequences. In addition, trainees can learn in an interactive environment and feel more inclined to engage.

Employees may feel more willing to work hard if they can sense what they need to learn. AR training provides many training opportunities for them to practice until they perfect their skills. Furthermore, they can complete the training almost anywhere that they can use their AR device.

You can reduce the time trainees spend working with experts. Instead of working directly with clients, they can practice until they feel confident enough to handle the situation in person. In that sense, AR also improves the customer experience. They are more likely to interact with a fully-trained employee, so customer service may run smoother and faster.

Lastly, AR training can decrease training costs. While the equipment is a pricey investment, you can reuse it for all of your trainees. Additionally, most people have smartphones that work with AR training apps.

How do I get started with augmented reality?

If you’re a business leader in an innovation role and are looking at ways to adopt AR in a useful and productive way, there are many considerations to keep in mind when adopting the technology. First, identify the specific use case for AR. There are several ways to implement AR, and each has its merits.

In my opinion, the best-suited option for training is wearable AR. This is where the user’s hands are free as they wear an augmented viewpoint headset. The two main players in this segment are the Microsoft HoloLens or the RealWear HMT-1. Both headsets allow a full range of motion and give the user overlaid information in real-time to assist them during complex tasks or procedures.

Consider your training needs to determine the best solution. Do you need a large field of view (FOV) or the ability to move around easily? Do you want graphical overlays or textual ones? Would it be helpful for trainees to pull data from schematics or instructions and have that information relayed to their eye in AR order to work hands-free? Or do you want them to see an augmented overlay of information in the real world to aid whatever job or task they are training on?

In my experience, the HoloLens is better suited for complex visualizations and works well where there’s a need for high-fidelity graphical overlays. On the flip side, the RealWear HMT-1 is more of an informational diegesis that will aid the worker by supplying additional context, voiceovers, manuals or even diagrams all within a small picture-in-picture viewpoint.

Augmented reality can help employers accelerate training time, educate employees and enhance the workforce by keeping skills relevant. While it is costly to implement, AR can save time and money in the long run because of its efficiency and effectiveness. As more companies integrate AR training into their employment strategies, consider how you can apply it to your business.

When looking to adopt the technology, first consider the use case and then decide on the hardware that would best fit the specific job you are looking to train for. Do your research to build a business case internally to help make a more informed decision.

 




New Augmented Reality Applications Assist Astronaut Repairs To Space Station

The T2 Augmented Reality (T2AR) project demonstrates how station crew members can inspect and maintain scientific and exercise equipment critical to maintaining crew health and achieving research goals without assistance from ground teams.

To kick off the T2AR activities in orbit in April, astronaut Soichi Noguchi of the Japan Aerospace Exploration Agency (JAXA) was tasked with maintenance for one of the space station crew’s pieces of exercise equipment, the T2 Treadmill. The inspection procedure is typically available as a PDF document to be accessed on a computer or tablet, which can be hard to hold while also operating tools or flashlights or examining equipment in a tight space. This time, no extra handheld instructions or communication with ground teams in Mission Control were necessary since the information was all in plain sight. Using the Microsoft HoloLens augmented reality (AR) goggles and armed with novel procedure tracking software NASA developed, Noguchi had step-by-step guidance and cues to assist in the work without referring to a separate screen.

T2AR is the first in-space operational use of the HoloLens in combination with custom-built AR software, which enables an astronaut to perform unassisted maintenance and inspections on a major piece of crew support hardware. This investigation builds on the Sidekick experiment former NASA astronaut Scott Kelly conducted in 2016.

This novel demonstration used 3-D directional cues to direct the astronaut’s gaze to the proper work sites and displayed the procedure instructions. The device followed an astronaut’s verbal instructions to navigate procedures and displayed AR cues and procedure text over the hardware as appropriate for the procedure step being performed. The system also provided supplemental information, such as instructional videos and system overlays, to assist in performing the procedure.

“AR tools hold the promise of allowing us to pre-package guidance and expertise,” says International Space Station associate scientist Bryan Dansberry at Johnson. “The space station is the perfect platform to test out AR systems and refine these tools so they will be ready when future astronauts need them. Closer to home, these tests help to mature software and AR technology now so expertise and support are available in remote locations around the world.”

NASA astronaut and Expedition 65 Flight Engineer Megan McArthur wears the specialized Sidekick headset and tests using augmented reality aboard the International Space Station. Credit: NASA

Since that first activity with Noguchi, astronaut Thomas Pesquet of ESA (European Space Agency) and NASA astronaut Megan McArthur have also used the AR application aboard the space station. The tests provided researchers with information about how the technology does and does not work to assist crew inspection maintenance procedures. With the completion of this maintenance activity, nine more test sessions remain in the technology demonstration plan.

While this demonstration was currently limited to the critical T2 Treadmill, the platform is designed to be used across a wide variety of space station needs in the future, leading to increased crew efficiency and activity execution accuracy. In addition to use on the space station, this technology could aid astronauts on future journeys to the Moon and Mars, when communication delays are greater, by allowing them to conduct tasks without waiting for further direction from Earth.

NASA’s Advanced Exploration Systems division sponsors this technology demonstration aboard the microgravity laboratory to advance future human and robotic exploration missions as part of NASA’s Moon and Mars exploration approach.