Visualization of Temperature Sensor Data in AR for Emergency Response

The use of thermal imaging data from handheld devices for identification of bodies in smoke-filled spaces has been in use to save lives for decades. This topic is not intended to study that use case. Several companies have successfully integrated the thermal imaging sensors into commercial PPE (“smart helmets”) worn by emergency responders.

This research topic will instead focus on advancing the state of the art of thermal vision. There could be extension of the types and size of thermal imaging technology suitable for including in AR-assisted emergency responses equipment. Currently, thermal imaging produces data in only 2D. Research into visualizing the heat source in 3D will be beneficial. In addition, the research should explore different methods for users to interact with the data displayed on their helmets, new and novel methods of distinguishing between heat-emitting sources detected using thermal imaging and combining the information about a building’s floor plan with the thermal imaging to trace safe routes to use when reaching targets under time constraints.

Stakeholders

While the emergency responder and military use cases are the clear beneficiaries of this research topic, there are also applications in automotive, such as displaying to a driver moving or stationary wildlife along a road side or forklift drivers “seeing” the thermal image of pedestrians in a warehouse or in a manufacturing environment. Additional imaging could prevent hazardous interactions. Similarly, when mounted in field equipment, the AR display can alert a driver to the presence of people or animals in the path of the equipment. When the technology is miniaturized and highly reliable, it may be used in enterprise AR use cases including but not limited to medical industry or animal husbandry for diagnostics and therapeutics. If the cost and size of the technologies is sufficiently low, the feature could be used for non-industrial applications, such as in games or education.

Possible Methodologies

The study of thermal imaging and design of new sensors for heat mapping will involve expertise in physics and engineering. Adapting new sensors to fit the AR display form factors for prototyping and user testing will require skills in electromechanical domains, semiconductors, and industrial design. In addition to research about thermal sensors, there will need to be research on user interaction with 3D thermal images.

Research Program

This topic can be combined with studies of other sensors that permit users to visualize through water, atmospheric conditions, and physical obstructions (e.g., walls). The study of sensor integration could also be extended to span visualization of a user’s vitals (heart beat, blood pressure, etc.) and 3D interactions.

Miscellaneous Notes

This topic was the focus of work by students at EPFL (Switzerland) in 2015. The students working on this project developed a thermal imaging smart visor in 2016. There was also a peer-reviewed journal paper published by one of the EPFL faculty working on the above project in 2018. In 2019, Darix, the start-up that was created out of the EPFL research, was acquired by Bullard. Longan Vision, another company in this field, has been adapting their system for Covid.

Keywords

Thermal imaging, 3D data visualization, smart helmets, Personal Protective Equipment, fire fighting, route tracing, 3D data interaction, public safety,

Research Agenda Categories

Industries, Displays, Technology

Expected Impact Timeframe

Medium

Related Publications

Using the words in this topic description and Natural Language Processing analysis of publications in the AREA FindAR database, the references below have the highest number of matches with this topic:

• Erickson, A., Kim, K., Schubert, R., Bruder, G., & Welch, G. (2019). Is It Cold in Here or Is It Just Me? Analysis of Augmented Reality Temperature Visualization for Computer-Mediated Thermoception. 2019 IEEE International Symposium on Mixed and Augmented Reality (ISMAR).
• Kang, S., Choi, H., Park, S., Park, C., Lee, J., Lee, U., & Lee, S.-J. (2019). Fire in Your Hands. The 25th Annual International Conference on Mobile Computing and Networking.
• Hirai, S., & Miki, N. (2019). Thermal Sensation Display with Controllable Thermal Conductivity. 2019 20th International Conference on Solid-State Sensors, Actuators and Microsystems & Eurosensors XXXIII (TRANSDUCERS & EUROSENSORS XXXIII).
• Mady, A. S., & Abou El-Seoud, S. (2020). An Interactive Augmented Reality Volume Rendering Mobile Application. Advances in Intelligent Systems and Computing, 888-896.
• Kraus, V., & Uzun, Y. (2020). Supporting Medical Auxiliary Work: The Central Sterile Services Department as a Challenging Environment for Augmented Reality Applications. 2020 IEEE International Symposium on Mixed and Augmented Reality (ISMAR).

More publications can be explored using the AREA FindAR research tool.

Author

Christine Perey

Last Published (yyyy-mm-dd)

2021-08-31

Go to Enterprise AR Research Topic Interactive Dashboard