Virtual User Interface

This use case pertains to using Augmented Reality to provide a user with an interactive interface for the real time operation and control of connected instruments or machines.

Prior to AR Adoption

Virtual User Interfaces permit a user to control and/or modify the settings of a piece of equipment without there being a physical user interface or screen attached to the same equipment. A Virtual User Interface can appear on a 2D screen which is connected over a network to the equipment in controls.  A Virtual User Interface is frequently provided in a centralized control room where technicians manage many processes running in parallel on remote machines. If there are issues with any of the machines or equipment they manage, the technician is alerted and can, using command line or graphic user interfaces on their local screens, adjust parameters on (or shut down) a remote machine. The virtual user interface can also be running in an application running on a smart phone or tablet and accessible by technicians out of a control room.

Business Challenges AR Introduction Addresses

As the adoption of connected equipment increases, some manufacturers do not provide a physical user interface on the equipment, relying entirely on virtual user interfaces. For example, some modern commercial vehicles for farming have attachments to treat fields or plant seeds that can be controlled at a distance but provide no user interface.   In some circumstances, a technician performing configurations, repairs or maintenance on a machine with a physical user interface, needs to add a user interface by connecting a tablet or other instrument via a wired or wireless network connection.  However, in metals and mining, power and energy, and oil and gas industries some equipment cannot be connected to external user interfaces in order to reduce the risk of igniting explosions.

Use Case with AR

With an AR-enabled device, a technician can point at a machine without a physical interface and, upon recognition of the target machine, a virtual user interface appears without the need to connect via wired or wireless network. The technician can perform some functions with a virtual user interface, without touching the machine or equipment.  The type of AR display used by the technician for depends on many factors:

Need for technician to use both hands

• Room in the vicinity where the procedures are performed for another screen pointed directly at the work space
• Support for introducing new display devices (e.g., wearable AR, projection AR)

With AR-enabled Virtual User Interfaces, the user can move around in a facility and be sure to be controlling only the machine whose interface is triggered by visual detection (e.g., a marker) or proximity (e.g., user’s position with respect to the machine).  Another feature that an AR-enabled Virtual User Interface can support is the real time capture of the actions taken or issues encountered by on-site users. Then, the data captured can be used for compliance records or other purposes.

Common roles of Users

Technicians and/or users that need to operate, control, repair or maintain machines that do not have physical interfaces  Inspectors

Business Benefits:

The benefits of AR-enhanced Virtual User Interfaces can be measured as reduced time to complete the procedures with lower (or no) errors.