Dynamic Distribution of Sensing and Processing Capacity via Body-Area Networks

Deploying enterprise AR solutions via head-mounted displays is inherently advantageous when simultaneous use of both hands is required. However, currently HMD performance is limited due to technological constraints, such as battery life and heat dissipation needs. One solution is to offload and distribute sensing and processing functions dynamically across other devices worn by the user (smartwatches, exoskeletons, sensing devices etc.)–that is, to a network of external wearable devices and resources.

A network of computing and sensing resources would not only decrease the display processing requirements, it would also enrich data feeds available in a complete AR system. A distributed architecture composed of haptic inputs, auditory signals, vibration and other sensors could provide richer data about the wearer (e.g. pose/posture of different body parts) and the environment. For example, using more diverse sensors could increase the system’s ability to detect safety risks before they are clear to the user.

The inputs from more and more varied sensors should be studied for new opportunities. Power savings and a less-heavy headworn display are clear benefits, but the architecture presents many challenges. Having multiple, distributed sources of input or computational functions will require integrating, in real time, separate data streams into a cohesive model. In addition to the user’s personal body-worn network, the merging and integration of computational functions could be performed on the edge or in the cloud, leveraging 5G in some cases.

This topic will include in its scope defining and testing different approaches to distributed sensors on the user’s body and distributing processing capacity and assessing their impacts if any on device safety, weight, size, power consumption and cost, as well as calculating the utility/benefit ratio of different architectures.

Stakeholders

HMD manufacturers, AR display device designers, sensor developers, body area network manufacturers, OEM manufacturers, integrated solution and software developers.

Possible Methodologies

Studies of wearable sensors, low-latency body area network technologies, and sensor fusion processors and their synchronization will be necessary to examine all aspects of this research topic. A research platform composed of configurable head-mounted AR display components could be designed. Sensor data tracing systems may need to be tested or developed to provide a complete understanding of architectural choices.

Research Program

This topic is at the intersection of multiple domains including but not limited to sensor configuration and control, sensor registrations, data filtering, sensor fusion, user context capture, and body area networks. The result could contribute to design of new HMD architectures, high-performance, and network-based resources and services (eg., 5G).

Miscellaneous Notes

Studies of optical and IMU sensor fusion for AR HMDs date have been published as early as 2003. A recent publication on the topic of 6DOF pose proves that the approach is very reliable. The ILLIXR Consortium is developing a platform which, if extended to include body area networks, could be appropriate for experimentation with different architectures.

Keywords

Distributed computing architecture, body-worn mesh networks, body area network, sensor configuration, sensor control, head-mounted display device, wearables, cloud

Research Agenda Categories

Displays, Standards, Business, Technology

Expected Impact Timeframe

Long

Related Publications

Using the words in this topic description and Natural Language Processing analysis of publications in the AREA FindAR database, the references below have the highest number of matches with this topic:

• Koutitas, G., Jabez, J., Grohman, C., Radhakrishna, C., Siddaraju, V., & Jadon, S. (2018). Demo/poster abstract: XReality research lab – Augmented reality meets Internet of Things. IEEE INFOCOM 2018 – IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS)
• Sosin, A., & Itoh, Y. (2019). WARP: Contributional Tracking Architecture Towards a Worldwide Augmented Reality Platform. 2019 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct)
• Hoang, T. N., Ferdous, H. S., Vetere, F., & Reinoso, M. (2018). Body as a Canvas. Proceedings of the 2018 Designing Interactive Systems Conference
• Michael, M., Llorca, J., & Tulino, A. (2019). Approximation Algorithms for the Optimal Distribution of Real-Time Stream-Processing Services. ICC 2019 – 2019 IEEE International Conference on Communications (ICC)
• Cziva, R., & Pezaros, D. P. (2017). On the Latency Benefits of Edge NFV. 2017 ACM/IEEE Symposium on Architectures for Networking and Communications Systems (ANCS)

More publications can be explored using the AREA FindAR research tool.

Author

Peter Orban, Christine Perey

Last Published (yyyy-mm-dd)

2021-08-31

Go to Enterprise AR Research Topic Interactive Dashboard