Creating AR experiences using existing authoring environments is a time-consuming process even for highly-trained developers. One of the greatest hurdles is the specification of real world features to which AR assets are attached/anchored. An alternative to manually defining anchors for AR assets is to adapt the outputs of real world 3D capture systems used in enterprises to be used as anchors for AR.

New interfaces and data encodings for 3D capture systems to make those systems directly accessible in an AR authoring pipeline will streamline or automate the preparation of AR experiences based in part on 3D capture of enterprise environments. However, there are many different 3D capture systems. This research topic focuses on development of guidelines and/or standards that will define the formats for 3D real world mapping which can be used by commercial AR authoring platform developers and publishers.

Stakeholders

This research is relevant to AR experience developers, AR managers, AR authoring platform publishing companies, AR service providers,

Possible Methodologies

Requirements for this approach for authoring AR experiences will be compared with existing and new standards and/or extensions of existing standards to automate AR authoring. The gaps and requirements will more easily be provided to appropriate standards development organizations for future work. If and when new interfaces and standards are published, enterprise AR authoring software vendors will be able to use these to streamline the AR authoring pipelines in industry.

Research Program

This topic could be combined with other topics to increase efficiencies in AR asset and experience authoring, management and delivery to reduce time and costs of integration with existing authoring and data management systems and platforms.

Miscellaneous Notes

This topic was submitted for 8th and 9th AREA research projects and received high support.

Keywords

3D world capture, depth sensing, liDAR, AR experience authoring, AR assets, anchoring, world mapping, SLAM, simultaneous localization and mapping, standards, data preparation, object-oriented programming, open systems, authoring systems, shape recognition, feature extraction, 3d modeling

Research Agenda Categories

Standards, Technology

Expected Impact Timeframe

Near

Related Publications

Using the words in this topic description and Natural Language Processing analysis of publications in the AREA FindAR database, the references below have the highest number of matches with this topic:

• Cavallo, M., & Forbes, A. G. (2019). CAVE-AR: A VR Authoring System to Interactively Design, Simulate, and Debug Multi-user AR Experiences. 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR).
• Speicher, M., Hall, B. D., Yu, A., Zhang, B., Zhang, H., Nebeling, J., & Nebeling, M. (2018). XD-AR. Proceedings of the ACM on Human-Computer Interaction, 2(EICS), 1-24.
• Babu, N. S. C. (2017). Keynote 1: Internet of Things(IoT) and augmented reality for e-learning. 2017 5th National Conference on E-Learning & E-Learning Technologies (ELELTECH).
• Blattgerste, J., Renner, P., & Pfeiffer, T. (2019). Authorable augmented reality instructions for assistance and training in work environments. Proceedings of the 18th International Conference on Mobile and Ubiquitous Multimedia.
• Zigart, T., & Schlund, S. (2020). Evaluation of Augmented Reality Technologies in Manufacturing – A Literature Review. Advances in Human Factors and Systems Interaction, 75-82.

More publications can be explored using the AREA FindAR research tool.

Author

Christine Perey

Last Published (yyyy-mm-dd)

2021-08-31

Go to Enterprise AR Research Topic Interactive Dashboard

Existing AR display devices do not have sensors to detect gases, radiation or other elements in the user’s environment. For many use cases of interest to the oil and gas industry as well as, mining, emergency responder and chemical industries, there needs to be research into the types and integration of existing chemical sensing technologies with the AR display or wearable computing system or to develop standard interfaces with IoT sensors to display readings in real time.

This research topic is a concrete example of a more fundamental research topic: the detection of an AR system user’s context beyond what can be done with existing sensors (cameras, microphone, IMU, etc). The scope of this research can be narrow or large, depending on the support provided from commercial or public agencies.

Stakeholders

Developers, operators and employees working in places where invisible gases may pose a risk, regulatory agencies, compliance officers.

Possible Methodologies

A laboratory would need to be developed for controlled exposure to chemicals and radiation sources. The lab and platform for testing will have off-the-shelf sensors and/or the project may require development of lightweight and power efficient sensors that are effective as alternatives to cameras and existing vision-based environmental capture. The testing and development of 3D interaction modes when user’s environmental sensors detect unsafe conditions is a fundamental part of this research domain.

Research Program

This topic or theme of research overlap with the topic of visualizing conditions in the direction or on the path of any moving object in atmosphere that is opaque or under water. The outcomes of this research could also be applied in non-industrial use cases (e.g., pollution sensing). The same research topic could be combined with study of user interfaces and interaction paradigms for the visually-impaired community.

Miscellaneous Notes

In 2012, the U.S. Department of Energy Office of Scientific and Technical Information (OSTI) funded research conducted at Department of Nuclear Engineering & Radiological Sciences, University of Michigan, on this topic. Preliminary results were reported in a poster about visualization of radiation in AR. A 2017 presentation about research on related topics conducted at CERN using HoloLens in particle accelerator environments is also relevant.

Keywords

Sensors, human factors, environment, oil and gas, chemical, power and energy, radiation, hazardous materials, radioactivity, explosives, chemical hazards,, volatile organic compounds, hazardous materials, aromatic compounds, gases, hydrocarbons, indicators (chemical), radioactivity, chemical hazards, chemical detection, gas sensors

Research Agenda Categories

Industries, Technology

Expected Impact Timeframe

Medium

Related Publications

Using the words in this topic description and Natural Language Processing analysis of publications in the AREA FindAR database, the references below have the highest number of matches with this topic:

• Tagami, A., & Shen, Z. (2020). LESAR: Localization System for Environmental Sensors using Augmented Reality. 2020 IEEE 44th Annual Computers, Software, and Applications Conference (COMPSAC).
• Park, Y. J., Ro, H., Byun, J.-H., & Han, T.-D. (2019). Adaptive projection augmented reality with object recognition based on deep learning. Proceedings of the 24th International Conference on Intelligent User Interfaces: Companion.
• Huai, J., Zhang, Y., & Yilmaz, A. (2019). The Mobile AR Sensor Logger for Android and iOS devices. 2019 IEEE SENSORS.
• Babu, N. S. C. (2017). Keynote 1: Internet of Things(IoT) and augmented reality for e-learning. 2017 5th National Conference on E-Learning & E-Learning Technologies (ELELTECH).
• Koutitas, G., Jabez, J., Grohman, C., Radhakrishna, C., Siddaraju, V., & Jadon, S. (2018). Demo/poster abstract: XReality research lab – Augmented reality meets Internet of Things. IEEE INFOCOM 2018 – IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS).

More publications can be explored using the AREA FindAR research tool.

Author

Christine Perey

Last Published (yyyy-mm-dd)

2021-08-31

Go to Enterprise AR Research Topic Interactive Dashboard

Some complex instructions and illustrations used in training or on the job require greater time and cognitive load for workers to understand, retain, and use when displayed in planar mode (2D). Three-dimensional representations and spatial vision enabled by stereoscopy has been shown to increase comprehension of spatially relevant concepts and increases their encoding and retention in memory. Although technology today enables spatial vision, frequently it requires some level of compromise around performance, wearability, or resource requirements.

This research topic focuses on measuring the impact of binocular (vs monocular) vision on short- and long-term memory encoding (i.e., the process of changing sensory inputs into forms that are stored in the brain and anchored in such a way that enables effective retrieval).

Stakeholders

User experience designers, AR experience developers, human resources professionals, AR display designers, AR display manufacturers, researchers studying cognition and performance of users in the workplace

Possible Methodologies

The research topic would need experts to develop and use a combination of existing neuro-analytical tools (tools that measure neurological brain activity) and biometric tools that infer neurological responses by proxy. The former includes EEG, fMRI (functional MRI), fNIRS (functional near-infared spectroscopy) and steady state topography (SST), all of which directly measure brain activity related to specific brain functions. For instance, SST measures the speed of electrical activity on the surface of the brain, linking changes in certain areas to specific metrics like engagement and memory encoding. The latter includes eye tracking, facial coding, and biometric data like heart rate monitoring. This research will analyze data for broader interpretation, offering insights into the use and impacts of 3D spatial viewing with AR, compared to measurements made by technology like SST and fMRI.

Research Program

This research topic can be integrated with fundamental research on brain function. It could also be combined with studies of specific use cases in which the system recalls the users’ spatial vision strategies and enhances those selectively. User experience design would also benefit from studies of this and related neuro-analytical tools and topics.

Miscellaneous Notes

There has been research published on the topics of spatial vision and more specifically on AR and memory encoding. This 2019 article published in Frontiers in Human Neuroscience reports on research conducted using AR to assess the impact of gender on spatial vision and anxiety.

Keywords

Spatial vision, spatial memory, visual encoding, memory anchoring, spatial frequency, receptive Field, modulation transfer function, high spatial frequency, threshold

Research Agenda Categories

Displays, Technology, End User and User Experience

Expected Impact Timeframe

Near

Related Publications

Using the words in this topic description and Natural Language Processing analysis of publications in the AREA FindAR database, the references below have the highest number of matches with this topic:

• Yong, S., & Wang, H.-C. (2018). Using Spatialized Audio to Improve Human Spatial Knowledge Acquisition in Virtual Reality. Proceedings of the 23rd International Conference on Intelligent User Interfaces Companion.
• Caluya, N. R., Plopski, A., Ty, J. F., Sandor, C., Taketomi, T., & Kato, H. (2018). Transferability of Spatial Maps: Augmented Versus Virtual Reality Training. 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR).
• Pangestu, A., & Setyaningrum, W. (2020). Instructional media for space geometry based on augmented reality to improve students’ spatial reasoning. Journal of Physics: Conference Series, 1581, 012058.
• Vovk, A., Gasques Rodrigues, D., Wild, F., & Weibel, N. (2019). SIG. Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems.
• Fuste, A., & Schmandt, C. (2019). HyperCubes: A Playful Introduction to Computational Thinking in Augmented Reality. Extended Abstracts of the Annual Symposium on Computer-Human Interaction in Play Companion Extended Abstracts.

More publications can be explored using the AREA FindAR research tool.

Author

Peter Orban, Christine Perey

Last Published (yyyy-mm-dd)

2021-08-31

Go to Enterprise AR Research Topic Interactive Dashboard

During the performance of tasks or fulfilment of roles, an employee may be unaware of changes in their involuntary bodily functions such as blood pressure and heart rate. When aware of any unusual metrics, the user can choose or be prompted to take appropriate actions. Visual and/or auditory information captured in real time by body-worn sensors (e.g., watches) could capture posture, head flexion and extension, whether the arms are above the shoulders, and possibly even squatting. The system could provide feedback on static posture duration and/or frequency and suggest postural changes or breaks based on criteria that increase the risk of muscular skeletal disorders.

The sensor data can be provided to the AR display through a body-area network. This topic focuses on the low-latency transmission and processing of observations from body-worn sensors to the user’s AR display and the presentation of data and recommendations in a compact and actionable manner. The system design research should explore both automated modes (which are triggered upon the user’s functions reaching a threshold) as well as manually controlled modes (e.g., enabled by a user seeking to obtain vital statistics). There must also be research to ensure that any AR visualization system is secure and upholds all relevant user data privacy protection policies.

Stakeholders

Companies monitor and manage workplaces for their suitability to employees. This research will be valuable to all workplace health and safety professionals, wellness.

Possible Methodologies

Research on this topic includes testing and studying the features of body-worn biometric devices in professional settings. It will be necessary to design and build a body-area network supporting existing standards in order to interface with radios and technologies on AR display devices. The research will also require designing and studying the usability, efficacy, and performance of AR-enriched user interfaces for providing a user with only pertinent biometric data via an AR display.

Research Program

This research topic can be combined with other topics pertaining to automated alerting of users to risk conditions using the AR display. It can also extend research on biometrics, biofeedback, and behavior modification therapies.

Miscellaneous Notes

A peer-reviewed study of a biofeedback system combined with a VR display for managing emotional states was published in SIGGRAPH ’17 proceedings . As documented in this 2019 review of the state of the art of body-area network usage in healthcare, the technology is maturing. Combining body-area networks and AR in professional domains is an unexplored field that has high potential for impact. Similarly, biometrics and the use of biofeedback in the workplace are very large and active fields of research. However, to date, their intersection with Augmented Reality has not been documented in the peer-reviewed literature.

Keywords

Biometrics, body sensors, blood pressure, heart rate, body temperature, body-area network, biofeedback, user data privacy protection, visualisation, alerts, biofeedback,

Research Agenda Categories

End User and User Experience, Technology

Expected Impact Timeframe

Medium

Related Publications

Using the words in this topic description and Natural Language Processing analysis of publications in the AREA FindAR database, the references below have the highest number of matches with this topic:

• Hoang, T. N., Ferdous, H. S., Vetere, F., & Reinoso, M. (2018). Body as a Canvas. Proceedings of the 2018 Designing Interactive Systems Conference.
• Pohl, H., Dalsgaard, T.-S., Krasniqi, V., & Hornbæk, K. (2020). Body LayARs: A Toolkit for Body-Based Augmented Reality. 26th ACM Symposium on Virtual Reality Software and Technology.
• Cao, Y., Qian, X., Wang, T., Lee, R., Huo, K., & Ramani, K. (2020). An Exploratory Study of Augmented Reality Presence for Tutoring Machine Tasks. Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems.
• Ueda, Y., Asai, Y., Enomoto, R., Wang, K., Iwai, D., & Sato, K. (2017). Body cyberization by spatial augmented reality for reaching unreachable world. Proceedings of the 8th Augmented Human International Conference.
• Koutitas, G., Jabez, J., Grohman, C., Radhakrishna, C., Siddaraju, V., & Jadon, S. (2018). Demo/poster abstract: XReality research lab – Augmented reality meets Internet of Things. IEEE INFOCOM 2018 – IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS).

More publications can be explored using the AREA FindAR research tool.

Author

Christine Perey

Last Published (yyyy-mm-dd)

2021-08-31

Go to Enterprise AR Research Topic Interactive Dashboard

As the complexity and computational requirements of world capture, world analysis, scene management, rendering, and human interactions with AR experiences increase, a delicate balance must be struck so as to ensure that the useful life of a device between recharges is not too low. Power management may be addressed through a combination of different approaches, including increase in use of low power DSPs, off-loading some computational tasks to the edge of the network (off-device services), and increasing power storage capacity. There is another potentially powerful resource to address the duration of wearable AR display usage. This topic focuses on the research and development of novel methods to capture power from the user or the environment that could be transferred to the power storage system.

Specifically, the proposed topic will research, build, and test implementations of new methods of energy harvesting from sources that have not been used in prior wearable AR display systems. There will need to be studies of human movement (steps, arms, hands), solar sources for users that are outdoors, and chemical reactions that release energy. This topic will span a wide range of technologies, including analysis of the power requirements of each individual AR display component.

Stakeholders

All users of wearable AR display devices and those who manage their use in the workplace will benefit from longer duration between recharges. Introducing or integrating novel power production, transfer, and storage technologies will have impacts on display costs, which could affect the number of devices purchased.

Possible Methodologies

The research will be leveraging developments in physics, chemistry, and other sciences pertaining to energy production and combining those with deeper studies of wearable AR display device power use. In addition to theoretical calculations, there will need to be prototypes developed and tested to measure the efficacy and efficiencies of power capture, storage, and use with AR displays. Finally, the introduction of new energy production, transfer, and storage methods or systems will need to be carefully studied for their safety in the workplace.

Research Program

This research topic could be studied in different environmental conditions, such as indoors, outdoors, and in different temperatures. In addition, there will need to be studies of different use cases in which some users are actively moving throughout a work shift. The results of this research would also be very valuable for non-enterprise users and display devices.

Miscellaneous Notes

Although quite dated by today’s standards, one of the first studies focusing on this topic was performed by Dr. Jannick Rolland and Dr. Henry Fuchs to examine the pros and cons of these two display options in a surgical use case. The study was published in the journal “Presence” in 2000. The topic of mitigating parallax-related registration errors is a highly active field of study, as demonstrated by this article published in December 2020 in the Frontiers in Robotics and AI journal.

Keywords

Power use, power consumption, power production, AR device energy sources, energy capture, energy production, AR device energy transfer, energy consumption, energy management, electromechanical, solar energy, friction, power storage for AR displays, human factors, weight, usability, portability, computational efficiency, low power electronics, electric batteries, power consumption, power conversion, power aware computing

Research Agenda Categories

Displays, Technology

Expected Impact Timeframe

Long

Related Publications

Using the words in this topic description and Natural Language Processing analysis of publications in the AREA FindAR database, the references below have the highest number of matches with this topic:

• Malhotra, N., Mann, A., & Pandey, N. (2019). Design of Low Power ECRL Based Power Gated 4:2 Compressor. 2019 6th International Conference on Signal Processing and Integrated Networks (SPIN)
• Ro, H., Park, Y. J., Byun, J.-H., & Han, T.-D. (2019). Mobile device interaction using projector metaphor. Proceedings of the 24th International Conference on Intelligent User Interfaces: Companion.
• Alavikia, Z., & Shabro, M. (2019). Pragmatic Industrial Augmented Reality in Electric Power Industry. 2019 International Power System Conference (PSC).
• Sanches, S. R. R., Oizumi, M., Oliveira, C., Damasceno, E. F., & Sementille, A. C. (2017). Aspects of User Profiles That Can Improve Mobile Augmented Reality Usage. 2017 19th Symposium on Virtual and Augmented Reality (SVR).
• Speicher, M., Hall, B. D., Yu, A., Zhang, B., Zhang, H., Nebeling, J., & Nebeling, M. (2018). XD-AR. Proceedings of the ACM on Human-Computer Interaction, 2(EICS), 1-24.

More publications can be explored using the AREA FindAR research tool.

Author

Christine Perey

Last Published (yyyy-mm-dd)

2021-08-31

Go to Enterprise AR Research Topic Interactive Dashboard