Training

Training

This use case pertains to using Augmented Reality to develop any skills and knowledge or fitness that relate to specific workplace competencies and behaviors. There is a gradual migration of terminology away from “training.” Professionals in this domain increasingly refer to this use case as “skill development,” “employee development” or “workforce development” which encompasses skill mastery, knowledge of and experience applying policies, and behaviors. Regardless of what it is called, this use case can be a component of and combined with other use cases, including complex assembly, inspection, maintenance and remote assistance.

Prior to AR Adoption

Training is defined as having one or more of the following characteristics:

  • Involves delivery of programs/content designed to increase a user’s skills and competencies, retention of concepts, rules, or changing of attitudes and behaviors or fitness in one or more domains which, when retained, can be used without the support of instructions in the workplace.
  • Can be delivered using solely documentation and materials prepared in advance
  • Can integrate machine learning, competency tests and situational awareness to generate materials or digital assets customized in real time for the user
  • Can be advanced automatically based on assessments of the user’s level of retention
  • Can be performed individually or in a group, in some scenarios, while interacting with other users

Increasing employee skills and competencies, retention/application of concepts or rules and  changing attitudes and behaviors or fitness is a continuous process in all industries. Due to everyone having had different life experiences, prior training and different learning method affinities or aptitudes, individualized skill development programs have higher success metrics than general, “one size fits all,” programs. Individualized programs are more expensive to develop and deliver. Consequently, most skill development programs combine group learning and individualized components.   Skill development materials are frequently in digital format, based on standard operating procedures, documentation and business or industry policies, and used in conjunction with a learning management system that tracks individual program progress and completion levels.  When in a training program, an employee is not directly contributing to performance of tasks, hence, businesses must balance skill development with productivity targets. Once an employee reaches a high skill development level, they are more capable of performing tasks without error or waste, and may be qualified to perform a wider variety of tasks or procedures than a less skilled employee. An employee with more skill mastery is a more flexible resource for the business.

Business Challenges AR Introduction Addresses

Material preparation, training and certification of employees can be time consuming, reducing the overall productivity of a workforce. As task and product complexity and diversity increase, the requirements for training and/or documentation to assist an untrained professional to develop skills, competencies and knowledge expand. Furthermore, training in a simulated or classroom environment may not match the real world environment or circumstances in which the employee is expected to use their acquired knowledge or skills.   If an employee needs to perform a process for which they are not trained in advance, they may make errors, waste materials or time. The cost of downtime caused when employees are unable to complete tasks without assistance is compounded with the cost of having a qualified professional travel and be dedicated to one task when there may be other urgent projects or tasks that could also benefit from the expert’s attention.

Use Case with AR

Augmented Reality-enabled skill development can be delivered to groups or individuals. Digital learning materials delivered via an AR-enabled display can re-use learning assets stored in a corporate knowledge base. The user’s interactions and skill mastery can be captured using the AR system and the status stored in the user’s learning management system record.   With AR in an environment which the real machine or products are available, the learner can see and hear the real world circumstances and interact with equipment or other people directly, increasing the kinesthetic learning component. When connected to machines and sensors with live data streams which appear on the AR display in the user’s field of view, the learner can make judgements about the duration of the task, the correct level of tension or pressure needed and other real time metrics, avoiding errors and bad habits while learning.  An AR-enabled system can also be worn or used by an expert to create highly compelling and interactive learning materials for playback as video or by other AR-enabled display users. The expert can use gestures or other tools to annotate/enhance the existing training materials with text, graphics, animations or videos.   The type of AR display used when developing skills depends on many factors:

  • Need for use of both hands to practice or perform a skill
  • Room in the vicinity where the training is conducted for another screen pointed directly at the work space
  • Support for introducing new display devices (e.g., wearable AR, projection AR)

Another capability that an AR-enabled training system can support is the real time capture of the learner’s achievement of competency. Then, the recording or data about the competency level can be stored in the learning management system.

Common roles of Users

Any employee or user that needs to acquire or master workplace competencies, skills and certifications.

Business Benefits:

The benefits of AR-enhanced training include raising the employee’s value to the business through increasing workplace competencies, skills and certifications.  In some situations, having AR-enabled instructions can reduce or remove the need to train a user in advance of performing a skill or procedure for the first time. An AR-enabled instruction delivery system may substitute reduce or eliminate the time that would be required for the training.  In addition, there can be benefits due to lower training time or on-the-spot lessons, which can translate to higher overall productivity of employees who do not need to stop working to acquire new competencies, skills and certifications.

Requirements

AR Hardware Requirements: Wear Ability / Comfort - Assisted

Assisted – The device weight worn on the head shall range between 125 grams and 420 grams The device shall support the addition of prescription lenses or fit over users spectacles The device shall have the ability to automatically adjust to the users Inter Pupillary Distance (IPD) to assist with ensuring eye comfort.

AR Hardware Requirements: Visual Tracking – Mobile

Mobile - The device AR object positioning accuracy MUST be within +XXmm. The device MUST be able to scan QR code (of 2 by 2 inches) from a minimum of 5 feet from a + 60 degree off-axis. Note: Optical and/or software zooming maybe utilized.  

AR Hardware Requirements: On-board Storage – Projection

Projection - The device MUST have a minimum on-board memory storage of >128GB  to support the OS, advanced perception and applications Projection - The device MUST have a minimum on-board RAM of >8GB.

AR Hardware Requirements: On-board Storage – Augmented

Augmented - The device MUST have a minimum on-board memory storage of 128GB  to support the OS, advanced perception and applications Augmented - The device MUST have a minimum on-board RAM of 8GB.

AR Hardware Requirements: Inputs / Outputs: Sensors - Mobile

The device SHOULD have an IMU with accelerometer, gyrometer and magnetometer that provides information to the software about gaze, and position. The device SHOULD have a RGB camera to enable see-what-I-see, bar-code scanning and video capture. Mobile - The device COULD have a time of flight depth sensor to measure distance and depth of the world  

AR Hardware Requirements: Inputs / Outputs: Sensors - Augmented

The device SHOULD have an IMU with accelerometer, gyrometer and magnetometer that provides information to the software about gaze, and position. The device SHOULD have a RGB camera to enable see-what-I-see, bar-code scanning and video capture. Augmented - The device MUST have time of flight depth sensor, to measure distance and depth of the world Augmented - The device MUST have two or more world cameras to enable location mapping and world reconstruction. Augmented - The device MUST have two or more eye tracking cameras to enable eye comfort and advanced perception capabilities such as a method of gaze.  

AR Hardware Requirements: Inputs / Outputs: Perception - Augmented

The device SHOULD support voice control as a method for software user interface commands. Augmented - The device COULD support eye tracking as a method of gaze, for software user interface commands. Augmented - The device COULD support gestures as a method for software user interface commands.

AR Hardware Requirements: Inputs / Outputs: Audio - Augmented

Augmented - The device COULD have multiple speakers to enable spatial audio

AR Hardware Requirements: Inputs / Outputs: Augmented

Augmented - The device MUST support a minimum resolution of 1280x960

AR Hardware Requirements: Inputs / Outputs: Assisted

Assisted - The device MUST support a minimum resolution of 854x480

AR Hardware Requirements: Field of View - Augmented

Augmented - The device MUST provide a 3D view (the images for both eyes are fully overlapping, just offset to provide the perception of 3D).

Augmented - The device MUST provide a minimum 35 degree diagonal field of vision where AR content can be displayed. Augmented - The device SHOULD provide a minimum 50 degree diagonal field of vision where AR content can be displayed.

Augmented - The device SHOULD have a variable range of operation from 40 cm to infinity, with HW support for developers to utilize from 20cm

AR Hardware Requirements: Field of View – Projection

Projection - The device MUST provide a minimum 60 degree field of vision in both directions (horizontal, vertical) where AR content can be displayed.

AR Hardware Requirements: Environmental - Augmented

Augmented -  The device SHOULD work in ambient temperatures 10ºC to 30ºC, 5 to 95% relative humidity Augmented -  The device displays SHOULD provide visible (easy to read) displays while in full ambient light conditions. Augmented - Commercial devices SHOULD work in environmental requirements documented in IP20 or greater Augmented - Industrial devices MUST work in environmental requirements documented in IP53 or greater

AR Hardware Requirements: Environmental - Assisted

Assisted - The device MUST work in ambient temperatures 0ºC to 35ºC/32ºF to 95ºF, 5 to 95% relative humidity Assisted - The device SHOULD work in environmental requirements documented in IP64 or greater

AR Hardware Requirements: Battery Life - Assisted Reality HMD

Under typical use, the device battery life of an Assisted Reality HMD using an AR solution SHOULD be a minimum of 12 hours on standby and 6 minimum hours of usage. Under heavy use, the device battery life of an Assisted Reality HMD using an AR solution MUST be a minimum of 3 consecutive hours usage.

AR Hardware Requirements: Battery Life - Mobile and/or Tablet

Under typical use, the device battery life of a Mobile and/or Tablet using an AR solution SHOULD be a minimum of 24 hours on standby and 12 minimum hours usage. Under heavy use, the device battery life of a Mobile and/or Tablet using an AR solution MUST be a minimum of 4 consecutive hours usage.

AR Hardware Requirements: Inputs / Outputs: Controller

Augmented - The device may support connection to a 6DoF controller for high precision control. (1st party)

AR Hardware Requirements: Inputs / Outputs: Mouse / Touchpad

The device COULD support connection to a Bluetooth mouse / Touchpad style device.

AR Hardware Requirements: Inputs / Outputs: Perception

The device SHOULD support voice control as a method for software user interface commands.  

AR Hardware Requirements: Inputs / Outputs: Processing

CPU (per device category) GPU (per device category) To be defined

AR Hardware Requirements: Inputs / Outputs: Sensors

The device SHOULD have an IMU with accelerometer, gyrometer and magnetometer that provides information to the software about gaze, and position. The device SHOULD have a RGB camera to enable see-what-I-see, bar-code scanning and video capture.

AR Hardware Requirements: On-board Storage - Assisted

Assisted - The device MUST have a minimum on-board memory storage of 64GB  to support the OS, advanced perception and applications Assisted - The device MUST have a minimum on-board RAM of 4GB.  

AR Hardware Requirements: Safety

Industrial devices MUST be intrinsically safe. The device SHOULD be able to be worn while wearing the required safety headgear (hard hat or bump cap). The device SHOULD meet OSHA and MSHA requirements for safety glasses. The device SHOULD have the option to meet side shield safety requirements. Electromagnetic - to be defined

AR Hardware Requirements: Visual Tracking - Augmented

Augmented - The device SHOULD perform on-board computing of spatial maps Augmented - The device SHOULD allow developers to access the computed spatial maps for application and tracking sdk development Augmented - The device AR object positioning accuracy MUST be within +5mm The device MUST be able to scan QR code (of 2 by 2 inches) from a minimum of 5 feet from a + 60 degree off-axis. Note: Optical and/or software zooming maybe utilized.

AR Hardware Requirements: Wear Ability / Comfort - Augmented

Augmented - The device weight worn on the head shall range between 260 grams and 560 grams The device shall support the addition of prescription lenses or fit over users spectacles The device shall have the ability to automatically adjust to the users Inter Pupillary Distance (IPD) to assist with ensuring eye comfort.

AR Software Requirements: AR Content Support

3D Visual Content
  • The content generation and consumption tool MUST support open 3D model formats.
  • The content generation and consumption tool SHOULD support proprietary 3D model formats.
  • The content generation and consumption tool SHOULD support animations.
2D Visual Content
  • The content generation and consumption tool MUST support open 2D formats.
  • The content generation and consumption tool SHOULD support proprietary 2D formats.
Media Content
  • The content generation and consumption tool SHOULD support open video formats.
  • The content generation and consumption tool SHOULD support open audio formats.

AR Software Requirements: AR Content: SOP

User Privileges The solution SHOULD require acknowledgment a Standard Operating Procedure (SOP)

AR Software Requirements: AR Localization

Matching the user's current environment with a previously mapped environment, along with understanding the user's current placement inside that digital environment. The device MUST support AR localization The platform SHOULD support Fast localization via localization hinting (e.g., QR Code, ArUco, or Assisted GPS) The platform SHOULD support marker-based tracking (e.g., using ArUco, AprilTag, or QR Code) The platform COULD support model-based tracking (e.g., CAD, scanned models) The platform COULD support real-time spatial tracking (e.g. SLAM)

AR Software Requirements: Integration

The solution SHOULD share data across integrated systems (i.e. LMS, MES, ERP, PLM) The solution COULD share data across users (e.g., between users during shift changes, knowledge base repositories, etc.)

AR Software Requirements: Manageability - on device

Embedded The platform SHOULD support 3rd party MDM/UEM agents Embedded The platform SHOULD support virtual private networks (VPNs) Embedded The platform MUST manage or disable device settings Embedded The device COULD support locked task mode which locks user from accessing restricted applications and device settings

AR Software Requirements: Mobile Device Management (MDM) - enterprise control

The platform MUST be able to control the permission to access third-party developers access to sensors and cameras The platform MUST allow MDM to control direct install/side load The platform SHOULD manage or disable device settings The device SHOULD support multi-user logins The platform SHOULD support over-the-air (OTA) or similar device provisioning The platform SHOULD support application management via MDM The platform COULD support remote access via MDM

AR Software Requirements: OS Features

Embedded The device software MUST support a single user accounts per device Embedded The device software SHOULD support multiple user accounts per device Embedded/Application The device software SHOULD support a web browser Embedded The device SHOULD allow third-parties developers the ability to read and write stored digital content Embedded The device MUST support file transfer Embedded The device MUST allow third-party developer access to sensors and cameras Embedded The device SHOULD support direct install/side load Embedded The device SHOULD support GDPR

AR Software Requirements: Production Deployments

Application The application SHOULD be deployable into multiple environments including on-premises (local cloud) and public cloud Application The applications MUST support General Data Protection Regulation (GDPR) requirements Application The application COULD be deployable into the Hybrid Cloud Application/Embedded The device MUST support access while off-network The platform COULD support the requirements of the Open AR Cloud Association

AR Software Requirements: Security

Application - the device SHOULD at least adhere to IEE/UL P2933 Application - the platform SHOULD support 3rd Party VPN Application - the platform SHOULD support Zero Trust configurations Application - the platform SHOULD use X509 certificates Application - the platform COULD support AES 256 device encryption Authentication - the solution SHOULD authenticate the user

AR Software Requirements: Software Developer Kits (SDKs) and Tools

The platform SHOULD support commonly available desktop software development tools The native SDK MUST enable access to device-specific functions The SDK SHOULD support common available AR deployment tools (e.g. Unity, Unreal, etc) The SDK SHOULD support cross-device development (e.g., Windows, MacOS, and Linux) The platform COULD support a cross-platform open AR Cloud SDK (https://github.com/OpenArCloud) Only Augmented - The platform SHOULD be OpenXR compliant The SDK MUST allow third-party developer access to sensors and cameras

AR Software Requirements: Supporting Use Cases: Authoring Augmented Reality Content

The authoring solution MUST support AR localization The authoring tool MUST originate content in the digital environment

Tags

Assisted Augmented Device Tablet / phone 

Settings

Industries

Architecture Automotive Aviation and Aerospace Chemical Commercial Vehicle Construction and Engineering Education Emergency Response Farming Food and Beverage Government Healthcare Industrial Equipment Logistics Manufacturing Marine Medical Metals & Mining Oil & Gas Power & Energy Smart Cities Telecommunications Transportation Utilities