Automatic Detection and Obfuscation of Facial and/or Personal Data of People in AR User Vicinity

In or in proximity of an AR-enabled user’s workplace, and in the field of view of the AR display’s camera, there may be people who have not agreed to (or are not able to grant permission for) their facial features or any personal data to be included in the AR system’s live video stream. If the private information of a person is captured without their explicit permission, a company may be held responsible for storing and any future use of the data. To reduce potential liabilities, entities responsible for a workplace will seek to implement a component of an AR display that removes all unrecognized faces from their systems.

This research topic includes using automatic detection of faces in an AR camera’s field of view, determining if the detected facial features are among those of people who have granted their permission to be tracked, and if the face does not match any in the database of those who have granted permission, to automatically and continuously obfuscate the features.

Stakeholders

All stakeholders in corporate security organizations including but not limited to privacy managers, workplace policy managers and risk managers. In order for obfuscation to be implemented in commercial systems, the research would need to take into account the requirements of the AR display ecosystem players (e.g., OEMs, solutions providers, system integrators and independent software vendors) who could leverage the results.

Possible Methodologies

The research will identify and evaluate automatic facial detection and identification technologies for suitability on an AR display. This may include assessment of speed, reliability and computational complexity. Computer vision solutions that meet requirements will need to be compared and tested to ensure that they do not reduce display performance or ability to meet the requirements of the user’s primary AR use cases. This topic may also involve introducing systems that interrupt an AR-assisted process and prompt a user to change orientations and/or ask unauthorized people to move out of camera view.

Research Program

This topic can be combined with the study of automatic detection and management of other sensitive data types. For example, the license plates of cars, names of people appearing in semi-public places (e.g. postal box), and voices of people in proximity of the AR-enabled user could be selectively removed based on the local regulations and privacy policies. It could also be combined with the research [ra-Tauthentication5-biometric][on biometric authentication for AR display users].

Miscellaneous Notes

A 2014 peer-reviewed paper describes tests performed and highlights the implications of the convergence of face recognition technology and use of AR.

Keywords

Face detection, facial identification, obfuscation of region of interest, personal information, privacy, privacy protection, security, compliance, biometrics, data security, access protocols.

Research Agenda Categories

Technology, End User and User Experience, Business

Expected Impact Timeframe

Medium

Related Publications

Using the words in this topic description and Natural Language Processing analysis of publications in the AREA FindAR database, the references below have the highest number of matches with this topic:

• Knierim, P., Wo?niak, P. W., Abdelrahman, Y., & Schmidt, A. (2019). Exploring the Potential of Augmented Reality in Domestic Environments. Proceedings of the 21st International Conference on Human-Computer Interaction with Mobile Devices and Services.
• Lutz, R. R. (2018). Safe-AR: Reducing Risk While Augmenting Reality. 2018 IEEE 29th International Symposium on Software Reliability Engineering (ISSRE).
• Lebeck, K., Ruth, K., Kohno, T., & Roesner, F. (2018). Towards Security and Privacy for Multi-user Augmented Reality: Foundations with End Users. 2018 IEEE Symposium on Security and Privacy (SP).
• Darwish, S. M., Mohallel, A. A., & Emara, D. (2018). An Enhanced Registration and Display Algorithm for Medical Augmented Reality. 2018 14th International Computer Engineering Conference (ICENCO).
• Abdelrazeq, A., Kohlschein, C., & Hees, F. (2019). A Cloud Based Augmented Reality Framework – Enabling User-Centered Interactive Systems Development. Advances in Intelligent Systems and Computing, 417-422.

More publications can be explored using the AREA FindAR research tool.

Author

Christine Perey

Last Published (yyyy-mm-dd)

2021-08-31

Go to Enterprise AR Research Topic Interactive Dashboard