AREA: Tell us how you became interested in joining the AREA.

Dr. Chaparro: I first heard
about the AREA from Brian Laughlin at Boeing. Brian was my human factors
doctoral student when I was at Wichita State and we’ve kept in touch over the
years. I’ve seen the kinds of things he’s been working on at Boeing and how it
overlapped with my research interest in human/computer interaction and usability
and user experience. I saw an opportunity to pursue them further through the
AREA group.

AREA: Could you tell us more about your background as it relates to AR?

Dr. Chaparro: My background is
in the area of usability and user experience. I have worked with a number of
different companies and technologies focusing on implementing design principles
to make it as easy as possible for people to use devices and tools.

I became interested in AR when
Google Glass was introduced. I could see the potential in industries such as aviation,
medical, and consumer products. My initial interest with Glass was to use it as
a training tool for my students. I also worked with a colleague at Wichita
State to study user interactions with Glass versus a cell phone.

And then HoloLens came out, and
for a year and a half now, we have been exploring the user experience side of
HoloLens. We want to get an idea of how the average person experiences this
technology. For instance: What are some of the issues from a UX standpoint? The
gesturing, window manipulation, texting, voice input – all of these methods of
interaction bring usability and user experience issues to the human-technology
interaction. A lot of the literature is focused on the usability of a particular
app, but there is very little out there on the integration of multiple technologies,
working across a multitude of tasks at the same time, or task-switching between
the physical and augmented environment. That is my interest, and then seeing the
application of this to a variety of domains. I consult, for example, with
healthcare professionals who believe that AR has great potential. Whatever the
domain, there is going to be this core issue of usability that will determine whether
it takes off or not. Eventually, it comes down to the comfort and the seamlessness
of the user experience in the tasks that they are doing.

AREA: How do you expect to benefit from your membership in the AREA?

Dr. Chaparro: I see the AREA as
a fantastic mix of academic researchers and industries that are applying the
technology. Human factors is an applied field, so we’re always looking for
practical applications of the things we’re studying in the lab. So I see that
as a huge benefit of the AREA. Then we’ll benefit from the work of the various
committees. We’ve been participating in the Safety and the Research Committees,
and hopefully, the Human Factors Committee in the future. We need to understand
what the issues are, because any problem that an industry is having is a potential
research project for one of my students. And that’s the other benefit: to
recognize the needs of industries that will need to hire students that have
knowledge of this technology. We want to understand what those needs are so we
can build them into our curriculum if they are not already there.

AREA: Based on what you have learned so far, what do you see as the
major outstanding issue that needs to be addressed to make AR more usable to
the average person?

Chaparro: With these new
glasses and head-mounted devices, certainly comfort is an issue, especially in
industries where they will need to wear them for an extended period of time.
That’s going to be huge. And not from just a comfort standpoint but also visually
– going back and forth between the physical and augmented world and what that
experience is like.

AREA: In addition to the research projects you mentioned, what other
areas of AR are being explored at Embry-Riddle?

Chaparro: My colleague Dr. Joseph
Keebler has been conducting research related to marker-based AR in medical
training. His area of expertise is medical human factors, teams, and training, so
he is excited about the technology from both a training standpoint and as a
real-time use tool for high performing teams. The issue is that, while it
appears that this technology is great and effective, we really need more research
to demonstrate how and when it is working, and how to best integrate it into
modern day systems.

One challenge is that there’s
a novelty effect problem. For instance, there are research projects being done
that show AR is better for performing a task, but it is really hard to tease
away the novelty side of that. In other words – are people improving due to
increased learning from the AR system? Or is it simply the fact that it’s this
fascinating and visually impressive technology that is garnering people’s
interest and keeping them engaged? Joe and I are interested in how to structure
a study so that we are looking at the true effectiveness of the technology above
and beyond the effects of its potential novelty. Joe has published a few papers
on AR, including a chapter in the Cambridge Handbook of Workplace Training and
Employee Development (Keebler, Patzer, Wiltshire, & Fiore, 2017)[1].

Another one of our colleagues,
Dr. Alex Chaparro, has been working on the use of AR in transportation. For
example, AR has many applications in aviation, maintenance documentation, and
driving environments. His main interest is in the uses of AR and VR in these environments
to train individuals to perform complex tasks.

We also have a VR gaming lab.
Joe and I have also done some psychometric work on the validation of a new
satisfaction instrument for video games that we’re now trying to apply to the
AR world (Phan, Keebler, & Chaparro, 2016)[2]. We
definitely see the benefits of this technology and would like to see it
succeed.

[1] Keebler, J. R., Patzer, B. S., Wiltshire, T.
J., & Fiore, S. M. (2017). 12 Augmented Reality Systems in Training. The
Cambridge Handbook of Workplace Training and Employee Development, 278.

[2] Phan, M. H., Keebler, J. R., & Chaparro, B. S. (2016). The
development and validation of the game user experience satisfaction scale
(GUESS). Human Factors, 58(8),
1217-1247.

The AREA Research Committee recently distributed to members two deliverables produced as part of the organization’s third research project, Assessing Safety and Human Factors of AR in the Workplace. This groundbreaking, member-exclusive research project produced the first framework for assessing and managing safety and human factors risks when introducing AR in the workplace. In addition to a tool to support decision-making, members also received an in-depth report of findings based on primary research.

Through the knowledge of its members and detailed interviews and research conducted with the wider enterprise AR ecosystem, the AREA’s reusable framework will promote a consistent approach to assessing safety and human factors of AR solutions.

This research was undertaken by AREA member Manufacturing Technology Centre (MTC) and managed by AREA sponsor member Christine Perey of PEREY Research and Consulting.

“For the first time, AREA members have a framework that will enable them to consider important requirements from the perspectives of key project roles and at each stage of the AR project,” said Perey. “The framework and supporting report are invaluable tools, built on the experience and knowledge gained by members and the larger community through many AR projects.”

“Through a combination of desk research and interviews with experts in the enterprise AR field, we captured rich and comprehensive insights into best practices and potential issues to overcome in these previously under-researched areas,” noted Amina Naqvi of the MTC, the author of the framework and research paper.

“This is another great example of the value the AREA brings to its members and the wider enterprise AR ecosystem,” said Mark Sage, Executive Director of the AREA. “By working together and learning from our fellow members, we’ve been able to produce research results that bring real benefits, and help to reduce the barriers to adoption for AR projects.”

The AREA has prepared a free Executive Summary of the Best Practice Report and a case study for non-members, “Assessing AR for Safety and Usability in Manufacturing” to help companies in the AR ecosystem to adopt or design safer and more usable wearable AR solutions.

If you’d like access to these resources please follow the links below hereto download them.

This Executive Summary was written to help organizations in the AR ecosystem to adopt or design safer and more usable wearable AR solutions.

One of the barriers to widespread adoption is the limited understanding of the actual and potential safety risks the technologies present for users and assets within any workplace or industrial environment. Currently there is no consistent approach or methodology to assess or certify how safe a wearable AR system/platform is (or could be) for a user in an industrial environment and no formal regulations or standards for AR safety globally, regionally or by industry.

The full report is available only to AREA members. It reviews the general risk management cycle as a preface to describing a new “Safe AR Design Best Practice” methodology for enterprise AR.

• Learn more about AREA Membership.
• To download the Executive Summary, please fill out this short form:

Benefits attributed to use of Augmented Reality are not just marketing hype; they are borne out in studies over the past decade. Despite requirements that still impose costs and other obstacles on AR implementation in the enterprise, the studies reveal that having AR-assisted systems guide users in performing complex tasks and support their collaboration is beneficial to performance.

This article summarizes some academic research findings and explains how AR can improve performance for scenarios where an AR system guides users through assembly and maintenance tasks.

Studies

Many studies highlight the differences in task completion speeds and error rates between two groups of users, with one group using a device such as a head-mounted display or watching a screen with AR instructions, and the other relying on traditional media such as a paper manual to complete tasks.

In four such studies, both groups performed identical, but relatively simple assembly and maintenance tasks. The table below summarizes the tasks on which the studies focus and the type of AR tested.

The studies found statistically significant differences in the performance of users, with AR-enabled groups having the edge. To cite Augmented Reality Efficiency in Manufacturing Industry: A Case Study:

• The group using AR instructions completed tasks 13% faster on average than the group using paper
• When using paper, the probability of using inappropriate tools was six times higher than with AR
• Also when using paper, the probability of putting a part in the wrong place was twice as high than with AR

These are the findings of one study and readers are invited to peruse the selection of linked studies above for specific information.

How AR Impacts Task Performance

The cited studies amply describe the positive impact of Augmented Reality on task performance, but how exactly does AR work?

Workers see instructions precisely overlaid on, or associated with, the parts to be handled or manipulated. These instructions can take the form of graphics, text or even audio. By delivering instructions when and where they’re needed, Augmented Reality reduces the cognitive work of part and tool recognition and allows users to concentrate more fully on the task at hand.

Furthermore, AR:

• Reduces body movements—workers tend to move around less when all the information is in one place
• Reduces attention switching—no need to switch between doing tasks and thumbing through a manual
• Promotes learning through spatial memory—it provides a frame of reference for fast and effective learning of new tasks, processes and equipment

Overall, Augmented Reality reduces both physical and cognitive efforts which makes for more efficient task completion than with traditional media such as manuals or on-screen help.

Caveat of AR

As any coin has two sides, these studies also have raised shortcomings associated with AR-assisted processes. A user’s exclusive focus on one area of view may reduce situational awareness of the periphery. This is known in industrial literature as “attention tunneling,” and is of acute concern in fighter pilots using head up displays.

Some research in Augmented Reality has uncovered potential issues with attention tunneling emerging from excessive focus on a single task and overreliance on AR cues. As one study mentions, this is primarily a design issue:

“Designers seeking to make use of the performance gains of AR systems also need to consider how the user manages their attention in such systems and avoid the over-reliance on cues from the AR system.”

Conclusion

Research into human factors of Augmented Reality reveals valuable findings that can be applied directly to the design of AR-assisted procedures for enterprise. The studies conclude that users can complete assembly and maintenance tasks more rapidly and with fewer mistakes with Augmented Reality. These conclusions will have significant impacts on business process design and operational costs.

Which human factors studies have you found helpful to guide your AR project design?

In terms of collaborative robotics, the widespread adoption of robots in historically manual manufacturing environments (which are subject to high product turnover, short production runs, and high variability in equipment configurations) is limited by the robots’ inability to effectively and safely integrate and interact with the existing human labor. Instead, so-called collaborative robots are relegated to secluded operations with minimal contact with the workforce. The robots’ inability to communicate with, understand the intention of, and establish a mutual understanding of the environment and situation with human coworkers decreases the robots’ usefulness in collaborative teams consisting of both robots and people. This limitation is driven by both the absence of tools and protocols needed for effectively describing and measuring human-robot interactions, an incomplete collection of metrics for assessing human-robot teaming performance, and insufficient protocols for enabling more intuitive interfacing with robotic tools. These challenges are compoudned when augmented reality technologies are used at the interface between the robotics and human workers.

This research topic focuses on providing the methods, protocols, and metrics necessary to evaluate the interactive and teaming capabilities of robot systems. It uses a task-driven decomposition of manufacturing processes to assess and assure the safety and effectiveness of human-robot collaborative teams.

Stakeholders

Manufacturers will benefit from the products generated as a result from this research project. Robotics providers can also benefit in that standard testing protocols for human-robot interaction will generate new sales tactics. End users will benefit in that the end state will be much safer in complex manufacturing environments.

Possible Methodologies

This collection of methods, protocols, and metrics will enable integrators and end-users to maximize the effectiveness and efficiency of collaborative human-robot teams in production processes, impacting both large-scale companies designing and repurposing hybrid manufacturing workflows, and smaller companies looking to begin introducing automated tools into manual processes.

Research Program

This research topic mirrors an existing project at NIST. Inspiration can be driven from the existing work generated by that team. Furthermore, IEEE is a leader in curating academic work in this area. Refer to IEEE RAS for related publication venues, including IEEE CASE, IEEE ICRA, and IEEE IROS.

Miscellaneous Notes

This topic requires significant hardware, middleware, and software integration. One open source framework is ROS-Industrial

Keywords

Robotics, human-robot interaction, human-computer interaction, remote monitoring, remote control, collaborative robots, autonomous agents, communication, computer vision, control systems, cooperative systems, grippers, human factors, human-robot interaction, industrial robots, industry 4.0, intelligent robots, multi-robot systems, occupational safety, robotics, safety

Research Agenda Categories

Standards, Technology, End User and User Experience

Expected Impact Timeframe

Long

Related Publications

Using the words in this topic description and Natural Language Processing analysis of publications in the AREA FindAR database, the references below have the highest number of matches with this topic:

• Muhammad, F., Hassan, A., Cleaver, A., & Sinapov, J. (2019). Creating a Shared Reality with Robots. 2019 14th ACM/IEEE International Conference on Human-Robot Interaction (HRI). </li>
• Jana Jost; Thomas Kirks; Preity Gupta; Dennis Lünsch; Jonas Stenzel (2018). Safe Human-Robot-Interaction in Highly Flexible Warehouses using Augmented Reality and Heterogenous Fleet Management System. 2018 IEEE International Conference on Intelligence and Safety for Robotics (ISR).
• Bolano, G., Juelg, C., Roennau, A., & Dillmann, R. (2019). Transparent Robot Behavior Using Augmented Reality in Close Human-Robot Interaction. 2019 28th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN).
• Whitsell, B., & Artemiadis, P. (2017). Physical Human-Robot Interaction (pHRI) in 6 DOF With Asymmetric Cooperation. IEEE Access, 5, 10834-10845.
• Xue, C., Qiao, Y., & Murray, N. (2020). Enabling Human-Robot-Interaction for Remote Robotic Operation via Augmented Reality. 2020 IEEE 21st International Symposium on “A World of Wireless, Mobile and Multimedia Networks” (WoWMoM).

More publications can be explored using the AREA FindAR research tool.

Author

Bill Bernstein

Last Published (yyyy-mm-dd)

2021-08-31

Go to Enterprise AR Research Topic Interactive Dashboard