Guidance

Guidance

This use case pertains to contextually-triggered AR-enhanced step-by-step instructions for any task.

There are aspects of this use case category that overlap with assembly, remote assistance and navigation use cases.

Prior to AR Adoption

Guidance is defined as having one or more of the following characteristics:

  • The order in which a user performs tasks is crucial or highly important to the desired outcome
  • Involves performance of steps or use or selection of specific parts that are unique to the process
  • Includes steps that involve one or more tools that could be confused
  • Components or tools must be examined and their conformance to criteria confirmed by the user
  • Requires components or tools that resemble one another but are unique or infrequently used
  • Requires using a tool which is not located at the same location as where the work will be done (hence, the user may need guidance to obtain)
  • The user may need to change locations to perform the task(s).

Even if a process is highly repetitive, there are some conditions in which a check list or step-by-step guide reduces uncertainty, errors and time to resolve doubt or correct errors. Instructions or an itemized list (e.g., pieces to be put in a unique customer order) can be provided to the user on paper (printed) or on a screen. In addition, while on the job, employees may need to perform tasks central to roles other than the one or few for which they are best qualified. In some circumstances, these “out of role” tasks may need to be done with detailed instructions without delay or no special training is required.

Business Challenges AR Introduction Addresses

Without Augmented Reality, getting the correct instructions and/or list of steps to the user at the right time with clarity and precision, is challenging. The user cannot carry or necessarily have full access to all documentation at all times. Further, the user may need to use new tools, or tools that are not immediately available. The user may need assistance to find, calibrate, operate the machine or tool which the instructions require the user to use. Finally, there are many objects in the physical world that resemble one another, but are, in fact and upon closer inspection are different. Without a strong and reliable system that automatically recognizes and tracks physical parts and tools, human errors are more frequent. In order to reduce errors, technicians must closely inspect all components. When the objects or components they choose are confirmed as being precisely the same as that called for in the instructions, the user may continue. Close examination and confirming choices takes time.

Use Case with AR

For guidance use cases, the user may have prior training about or experience performing a task or process, but receives a highly contextual guide as if a mentor were at their side. If using new tools or performing an unfamiliar process, the use of AR-enhanced guidance can confirm completion of a step before prompting the user to the next step. Due to the breadth of industries and high number of specific use cases that fit this category, it is not possible to describe all the different variations in which guidance offers an employee support. AR guide experiences can be stand-alone (running entirely on the user’s device), or connected via a network with company documentation and standard operating procedures. When the AR guidance system is integrated with operational systems including connected machines, supply chain management and order fulfillment systems, the employee does not need to intera directly with those services and explicitly log a task. In some conditions, the employees with high performance metrics can also become, with AR-assisted capture technology, the expert and record new instructions for others than will perform the same or similar tasks in the future. When the AR system is connected to the PLM (Product Lifecycle Management) system, any changes made to the inspection process can be rolled out easily and quickly to the entire workforce.

Common roles of Users

  • Any professional who needs to receive and to follow precise instructions

Business Benefits:

The primary benefits of AR-assisted guidance are the reduction in time and errors when performing a task or procedure. Other benefits include other measures of employee productivity and satisfaction. In some instances, particularly when the instructions are unique, AR-assisted guidance take the place of needing to remember a part or step and can, consequently, reduce a user’s cognitive load. In addition, there can be benefits due to lower training time and higher overall productivity of employees, even if they have not been trained on the specific tasks.

Requirements

AR Hardware Requirements: Wear Ability / Comfort - Assisted

Assisted – The device weight worn on the head shall range between 125 grams and 420 grams The device shall support the addition of prescription lenses or fit over users spectacles The device shall have the ability to automatically adjust to the users Inter Pupillary Distance (IPD) to assist with ensuring eye comfort.

AR Hardware Requirements: Visual Tracking – Mobile

Mobile - The device AR object positioning accuracy MUST be within +XXmm. The device MUST be able to scan QR code (of 2 by 2 inches) from a minimum of 5 feet from a + 60 degree off-axis. Note: Optical and/or software zooming maybe utilized.  

AR Hardware Requirements: On-board Storage – Projection

Projection - The device MUST have a minimum on-board memory storage of >128GB  to support the OS, advanced perception and applications Projection - The device MUST have a minimum on-board RAM of >8GB.

AR Hardware Requirements: On-board Storage – Augmented

Augmented - The device MUST have a minimum on-board memory storage of 128GB  to support the OS, advanced perception and applications Augmented - The device MUST have a minimum on-board RAM of 8GB.

AR Hardware Requirements: Inputs / Outputs: Sensors - Mobile

The device SHOULD have an IMU with accelerometer, gyrometer and magnetometer that provides information to the software about gaze, and position. The device SHOULD have a RGB camera to enable see-what-I-see, bar-code scanning and video capture. Mobile - The device COULD have a time of flight depth sensor to measure distance and depth of the world  

AR Hardware Requirements: Inputs / Outputs: Sensors - Augmented

The device SHOULD have an IMU with accelerometer, gyrometer and magnetometer that provides information to the software about gaze, and position. The device SHOULD have a RGB camera to enable see-what-I-see, bar-code scanning and video capture. Augmented - The device MUST have time of flight depth sensor, to measure distance and depth of the world Augmented - The device MUST have two or more world cameras to enable location mapping and world reconstruction. Augmented - The device MUST have two or more eye tracking cameras to enable eye comfort and advanced perception capabilities such as a method of gaze.  

AR Hardware Requirements: Inputs / Outputs: Perception - Augmented

The device SHOULD support voice control as a method for software user interface commands. Augmented - The device COULD support eye tracking as a method of gaze, for software user interface commands. Augmented - The device COULD support gestures as a method for software user interface commands.

AR Hardware Requirements: Inputs / Outputs: Audio - Augmented

Augmented - The device COULD have multiple speakers to enable spatial audio

AR Hardware Requirements: Inputs / Outputs: Augmented

Augmented - The device MUST support a minimum resolution of 1280x960

AR Hardware Requirements: Inputs / Outputs: Assisted

Assisted - The device MUST support a minimum resolution of 854x480

AR Hardware Requirements: Field of View - Augmented

Augmented - The device MUST provide a 3D view (the images for both eyes are fully overlapping, just offset to provide the perception of 3D).

Augmented - The device MUST provide a minimum 35 degree diagonal field of vision where AR content can be displayed. Augmented - The device SHOULD provide a minimum 50 degree diagonal field of vision where AR content can be displayed.

Augmented - The device SHOULD have a variable range of operation from 40 cm to infinity, with HW support for developers to utilize from 20cm

AR Hardware Requirements: Field of View – Projection

Projection - The device MUST provide a minimum 60 degree field of vision in both directions (horizontal, vertical) where AR content can be displayed.

AR Hardware Requirements: Environmental - Augmented

Augmented -  The device SHOULD work in ambient temperatures 10ºC to 30ºC, 5 to 95% relative humidity Augmented -  The device displays SHOULD provide visible (easy to read) displays while in full ambient light conditions. Augmented - Commercial devices SHOULD work in environmental requirements documented in IP20 or greater Augmented - Industrial devices MUST work in environmental requirements documented in IP53 or greater

AR Hardware Requirements: Environmental - Assisted

Assisted - The device MUST work in ambient temperatures 0ºC to 35ºC/32ºF to 95ºF, 5 to 95% relative humidity Assisted - The device SHOULD work in environmental requirements documented in IP64 or greater

AR Hardware Requirements: Battery Life - Assisted Reality HMD

Under typical use, the device battery life of an Assisted Reality HMD using an AR solution SHOULD be a minimum of 12 hours on standby and 6 minimum hours of usage. Under heavy use, the device battery life of an Assisted Reality HMD using an AR solution MUST be a minimum of 3 consecutive hours usage.

AR Hardware Requirements: Battery Life - Mobile and/or Tablet

Under typical use, the device battery life of a Mobile and/or Tablet using an AR solution SHOULD be a minimum of 24 hours on standby and 12 minimum hours usage. Under heavy use, the device battery life of a Mobile and/or Tablet using an AR solution MUST be a minimum of 4 consecutive hours usage.

AR Hardware Requirements: Inputs / Outputs: Controller

Augmented - The device may support connection to a 6DoF controller for high precision control. (1st party)

AR Hardware Requirements: Inputs / Outputs: Mouse / Touchpad

The device COULD support connection to a Bluetooth mouse / Touchpad style device.

AR Hardware Requirements: Inputs / Outputs: Perception

The device SHOULD support voice control as a method for software user interface commands.  

AR Hardware Requirements: Inputs / Outputs: Processing

CPU (per device category) GPU (per device category) To be defined

AR Hardware Requirements: Inputs / Outputs: Sensors

The device SHOULD have an IMU with accelerometer, gyrometer and magnetometer that provides information to the software about gaze, and position. The device SHOULD have a RGB camera to enable see-what-I-see, bar-code scanning and video capture.

AR Hardware Requirements: On-board Storage - Assisted

Assisted - The device MUST have a minimum on-board memory storage of 64GB  to support the OS, advanced perception and applications Assisted - The device MUST have a minimum on-board RAM of 4GB.  

AR Hardware Requirements: Safety

Industrial devices MUST be intrinsically safe. The device SHOULD be able to be worn while wearing the required safety headgear (hard hat or bump cap). The device SHOULD meet OSHA and MSHA requirements for safety glasses. The device SHOULD have the option to meet side shield safety requirements. Electromagnetic - to be defined

AR Hardware Requirements: Visual Tracking - Augmented

Augmented - The device SHOULD perform on-board computing of spatial maps Augmented - The device SHOULD allow developers to access the computed spatial maps for application and tracking sdk development Augmented - The device AR object positioning accuracy MUST be within +5mm The device MUST be able to scan QR code (of 2 by 2 inches) from a minimum of 5 feet from a + 60 degree off-axis. Note: Optical and/or software zooming maybe utilized.

AR Hardware Requirements: Wear Ability / Comfort - Augmented

Augmented - The device weight worn on the head shall range between 260 grams and 560 grams The device shall support the addition of prescription lenses or fit over users spectacles The device shall have the ability to automatically adjust to the users Inter Pupillary Distance (IPD) to assist with ensuring eye comfort.

AR Software Requirements: AR Content Support

3D Visual Content
  • The content generation and consumption tool MUST support open 3D model formats.
  • The content generation and consumption tool SHOULD support proprietary 3D model formats.
  • The content generation and consumption tool SHOULD support animations.
2D Visual Content
  • The content generation and consumption tool MUST support open 2D formats.
  • The content generation and consumption tool SHOULD support proprietary 2D formats.
Media Content
  • The content generation and consumption tool SHOULD support open video formats.
  • The content generation and consumption tool SHOULD support open audio formats.

AR Software Requirements: AR Content: SOP

User Privileges The solution SHOULD require acknowledgment a Standard Operating Procedure (SOP)

AR Software Requirements: AR Localization

Matching the user's current environment with a previously mapped environment, along with understanding the user's current placement inside that digital environment. The device MUST support AR localization The platform SHOULD support Fast localization via localization hinting (e.g., QR Code, ArUco, or Assisted GPS) The platform SHOULD support marker-based tracking (e.g., using ArUco, AprilTag, or QR Code) The platform COULD support model-based tracking (e.g., CAD, scanned models) The platform COULD support real-time spatial tracking (e.g. SLAM)

AR Software Requirements: Integration

The solution SHOULD share data across integrated systems (i.e. LMS, MES, ERP, PLM) The solution COULD share data across users (e.g., between users during shift changes, knowledge base repositories, etc.)

AR Software Requirements: Manageability - on device

Embedded The platform SHOULD support 3rd party MDM/UEM agents Embedded The platform SHOULD support virtual private networks (VPNs) Embedded The platform MUST manage or disable device settings Embedded The device COULD support locked task mode which locks user from accessing restricted applications and device settings

AR Software Requirements: Mobile Device Management (MDM) - enterprise control

The platform MUST be able to control the permission to access third-party developers access to sensors and cameras The platform MUST allow MDM to control direct install/side load The platform SHOULD manage or disable device settings The device SHOULD support multi-user logins The platform SHOULD support over-the-air (OTA) or similar device provisioning The platform SHOULD support application management via MDM The platform COULD support remote access via MDM

AR Software Requirements: OS Features

Embedded The device software MUST support a single user accounts per device Embedded The device software SHOULD support multiple user accounts per device Embedded/Application The device software SHOULD support a web browser Embedded The device SHOULD allow third-parties developers the ability to read and write stored digital content Embedded The device MUST support file transfer Embedded The device MUST allow third-party developer access to sensors and cameras Embedded The device SHOULD support direct install/side load Embedded The device SHOULD support GDPR

AR Software Requirements: Production Deployments

Application The application SHOULD be deployable into multiple environments including on-premises (local cloud) and public cloud Application The applications MUST support General Data Protection Regulation (GDPR) requirements Application The application COULD be deployable into the Hybrid Cloud Application/Embedded The device MUST support access while off-network The platform COULD support the requirements of the Open AR Cloud Association

AR Software Requirements: Security

Application - the device SHOULD at least adhere to IEE/UL P2933 Application - the platform SHOULD support 3rd Party VPN Application - the platform SHOULD support Zero Trust configurations Application - the platform SHOULD use X509 certificates Application - the platform COULD support AES 256 device encryption Authentication - the solution SHOULD authenticate the user

AR Software Requirements: Software Developer Kits (SDKs) and Tools

The platform SHOULD support commonly available desktop software development tools The native SDK MUST enable access to device-specific functions The SDK SHOULD support common available AR deployment tools (e.g. Unity, Unreal, etc) The SDK SHOULD support cross-device development (e.g., Windows, MacOS, and Linux) The platform COULD support a cross-platform open AR Cloud SDK (https://github.com/OpenArCloud) Only Augmented - The platform SHOULD be OpenXR compliant The SDK MUST allow third-party developer access to sensors and cameras

AR Software Requirements: Supporting Use Cases: Authoring Augmented Reality Content

The authoring solution MUST support AR localization The authoring tool MUST originate content in the digital environment

AR Use Case Requirements: Maintenance

The user MUST be able to provide feedback on the step completed The solution COULD be situation-aware The solution SHOULD be location aware

Tags

All of the above Assisted Augmented Device Projection Tablet / phone 

Settings

Industries

Architecture Automotive Aviation and Aerospace Chemical Commercial Vehicle Construction and Engineering Education Emergency Response Farming Food and Beverage Government Healthcare Industrial Equipment Logistics Manufacturing Marine Medical Metals & Mining Oil & Gas Power & Energy Smart Cities Telecommunications Transportation Utilities