Testing Protocols for AR-assisted Human-Robot Interaction

In terms of collaborative robotics, the widespread adoption of robots in historically manual manufacturing environments (which are subject to high product turnover, short production runs, and high variability in equipment configurations) is limited by the robots’ inability to effectively and safely integrate and interact with the existing human labor. Instead, so-called collaborative robots are relegated to secluded operations with minimal contact with the workforce. The robots’ inability to communicate with, understand the intention of, and establish a mutual understanding of the environment and situation with human coworkers decreases the robots’ usefulness in collaborative teams consisting of both robots and people. This limitation is driven by both the absence of tools and protocols needed for effectively describing and measuring human-robot interactions, an incomplete collection of metrics for assessing human-robot teaming performance, and insufficient protocols for enabling more intuitive interfacing with robotic tools. These challenges are compoudned when augmented reality technologies are used at the interface between the robotics and human workers.

This research topic focuses on providing the methods, protocols, and metrics necessary to evaluate the interactive and teaming capabilities of robot systems. It uses a task-driven decomposition of manufacturing processes to assess and assure the safety and effectiveness of human-robot collaborative teams.

Stakeholders

Manufacturers will benefit from the products generated as a result from this research project. Robotics providers can also benefit in that standard testing protocols for human-robot interaction will generate new sales tactics. End users will benefit in that the end state will be much safer in complex manufacturing environments.

Possible Methodologies

This collection of methods, protocols, and metrics will enable integrators and end-users to maximize the effectiveness and efficiency of collaborative human-robot teams in production processes, impacting both large-scale companies designing and repurposing hybrid manufacturing workflows, and smaller companies looking to begin introducing automated tools into manual processes.

Research Program

This research topic mirrors an existing project at NIST. Inspiration can be driven from the existing work generated by that team. Furthermore, IEEE is a leader in curating academic work in this area. Refer to IEEE RAS for related publication venues, including IEEE CASE, IEEE ICRA, and IEEE IROS.

Miscellaneous Notes

This topic requires significant hardware, middleware, and software integration. One open source framework is ROS-Industrial

Keywords

Robotics, human-robot interaction, human-computer interaction, remote monitoring, remote control, collaborative robots, autonomous agents, communication, computer vision, control systems, cooperative systems, grippers, human factors, human-robot interaction, industrial robots, industry 4.0, intelligent robots, multi-robot systems, occupational safety, robotics, safety

Research Agenda Categories

Standards, Technology, End User and User Experience

Expected Impact Timeframe

Long

Related Publications

Using the words in this topic description and Natural Language Processing analysis of publications in the AREA FindAR database, the references below have the highest number of matches with this topic:

• Muhammad, F., Hassan, A., Cleaver, A., & Sinapov, J. (2019). Creating a Shared Reality with Robots. 2019 14th ACM/IEEE International Conference on Human-Robot Interaction (HRI). </li>
• Jana Jost; Thomas Kirks; Preity Gupta; Dennis Lünsch; Jonas Stenzel (2018). Safe Human-Robot-Interaction in Highly Flexible Warehouses using Augmented Reality and Heterogenous Fleet Management System. 2018 IEEE International Conference on Intelligence and Safety for Robotics (ISR).
• Bolano, G., Juelg, C., Roennau, A., & Dillmann, R. (2019). Transparent Robot Behavior Using Augmented Reality in Close Human-Robot Interaction. 2019 28th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN).
• Whitsell, B., & Artemiadis, P. (2017). Physical Human-Robot Interaction (pHRI) in 6 DOF With Asymmetric Cooperation. IEEE Access, 5, 10834-10845.
• Xue, C., Qiao, Y., & Murray, N. (2020). Enabling Human-Robot-Interaction for Remote Robotic Operation via Augmented Reality. 2020 IEEE 21st International Symposium on “A World of Wireless, Mobile and Multimedia Networks” (WoWMoM).

More publications can be explored using the AREA FindAR research tool.

Author

Bill Bernstein

Last Published (yyyy-mm-dd)

2021-08-31

Go to Enterprise AR Research Topic Interactive Dashboard