Remote Assistance

This use case pertains to having remote subject matter experts assisting on-site technicians or users (together called “on-site users”) in performance of any task or process which would otherwise not be completed (on time or according to guidelines). There are aspects of this use case category that overlap with guidance, assembly and navigation use cases. It may also include an inspection use case (also known as “remote inspection”).

Prior to AR Adoption

Remote Assistance is defined as having one or more of the following characteristics:

• Involves two or more users communicating in real time in which an on-site user shows the issue and can send other signals (e.g., connected tools or diagnostic instruments) from their perspective to an expert (at a different location)
• An expert is chosen and available to provide real time instructions and feedback and/or to ask questions and guide the on-site user without requiring travel to the site
• Expert’s system can annotate/enhance the on-site user’s experience with graphics, animations and/or videos

In some cases, such as when repairing an unfamiliar machine or performing complex assembly procedures, an on-site user seeks the input, guidance, advice or approval of a remote expert. The expert may review photos sent from the on-site user’s computer or smartphone. They may receive data generated by the connected machine. They may also have a voice (telephone) conversation during which the on-site user describes issues encountered or asks for guidance.      If the on-site user is not sufficiently precise in describing the issue encountered, or an expert is unable to provide sufficiently clear guidance, the expert needs to travel to the same location.

Business Challenges AR Introduction Addresses

As task and product complexity and diversity increase, the requirements for training and/or documentation to assist an untrained professional (on-site user) expand. If situations occur where an on-site user is not trained in advance or lacks documentation to perform tasks required on site, or when there are unanticipated challenges, an expert is needed. There are also cases in which the parts or materials for an on-site procedure (e.g., a repair) must be examined and their compliance certified. In either situation, an on-site user may need the support of an expert.

Those in the field who need an expert must either wait for assistance to arrive (in person), or a task may be rescheduled or delayed until the expert is available.  If the machine is in a factory, the entire production may experience down time. If a machine is mobile, such as a commercial vehicle, it may not be possible to take it back to a garage.  Another example is when there are issues with supporting machinery, such as a farm vehicle that is treating fields or planting.

The cost of downtime caused when on-site users are unable to complete tasks without assistance is compounded with the cost and CO2 emissions of having an expert travel and be dedicated to one task when there may be other urgent projects or tasks that could also benefit from the expert’s attention.

Under conditions where a professional cannot go to or access a site in person, such as locations where there are restrictions due to Covid policies, an unskilled technician can run a local AR experience and follow instructions of the remote expert.

Training technicians and all on-site users can be time consuming, costly and reducing the overall productivity of the on-site user. It also restricts which employees are able to perform specific tasks (the flexibility of the workforce is lower).

Use Case with AR

When an on-site user encounters an issue or situation for which the documentation is incomplete or poor quality, or an inspection is required, an expert who is not local to the work site may be needed. With remote assistance, the on-site user can establish a communication session during which all the local conditions can be sent from the AR device in the field (e.g., audio and video) to an expert in another location.

The remote expert can see and hear the on-site user’s circumstances and questions from the point of view of the on-site user. An expert can use an AR-enabled system to annotate/enhance the on-site user’s experience with text, graphics, animations and/or videos. In some scenarios the instructions and annotations appear independent of the on-site user’s context; in others, the expert’s annotations may be registered to the objects in the user’s field of view.

The type of AR display used by the technician depends on many factors:

• Need for technician to use both hands
• Room in the vicinity where the procedures are performed for another screen pointed directly at the work space
• Support for introducing new display devices (e.g., wearable AR, projection AR)

With remote assistance, the on-site user is no longer required to take a product or machine to the expert, to wait or delay completion of a task, or to switch focus of attention between the task itself and the expert’s instructions. All of the actions and inputs from the on-site user’s location are seen and heard by the expert in real time until the procedure is completed or inspection finished.   Another capability that an AR-enabled system with remote assistance can support is the real time capture of the issues encountered by on-site users. Then, the recording can be used for future training purposes, or to record successful (or incomplete) procedures.

Common roles of Users

• Technicians
• Inspectors
• Users of machines that are untrained in diagnosing, repairing or maintaining their machines
• Experts with knowledge that is or may be needed by many users in the field/on-site

Business Benefits:

The benefits of AR-enhanced remote assistance can be measured as reduced time to complete the procedures with lower (or no) errors and need for travel. Also the on-site user will be able to perform and to learn how to complete more diverse and complex tasks without prior training, leading to higher workforce flexibility.   In addition, there can be benefits due to lower training time and higher overall productivity of experts who do not need to travel to support field technicians or even end users operating complex machinery for limited time.