Inspection

Inspection

This use case pertains to the use of AR for inspecting the results of processes, products and workplaces for safety and quality measurement or documentation. It helps detect and/or reduce incidence of a wide range of different types of risks and ensures that all requirements and policies are enforced. This use case includes other activities such as “monitoring” and “measurement” of physical objects and/or processes performed by an AR-enabled employee.

Prior to AR Adoption

  • Inspection is defined as having one or more of the following characteristics:
  • the different phases of a process or the result must meet criteria before being approved
  • the criteria for approving a process completion or final product can be determined visually, through an instrument or haptic feedback
  • the status and outcomes of the inspection must be recorded in one or more systems
  • a specialist, trained to the criteria and different from the person performing the task or producing the product, may be required to certify the status and outcomes

Many industries and companies have policies concerning the quality and/or completion of a process. Specifically, policies require that prior to advancing or considering the procedure or product complete, an independent and objective inspector review and sign off the work.   A qualified or trained inspector is not always at the location where a task is completed. In some cases, photographs of a product or current status of a procedure may be taken by an employee and sent to a remote inspector for review. While the inspector either travels to the point of inspection, or performs a virtual review and approval, the employee must wait for an independent authority’s approval (signoff) before proceeding.

Business Challenges AR Introduction Addresses

Without Augmented Reality-enabled inspection or remote assistance, the employee may need to wait for an independent and objective inspector to review and approve a task or product before continuing to another work order or repeating the tasks. This lowers employee productivity.   Inspectors are limited in number and are valuable resources which, without automated or AR-enabled systems may spend a portion of their time traveling between sites where they perform inspections for relatively short duration. Despite efforts to be consistent and diligent, inspectors may also differ in their application of criteria between one another or between days and times of review, introducing variability and potential risks. Finally, due to COVID policies, inspectors may not be allowed on site.

Use Case with AR

In this use case, a user’s AR-enabled device is equipped with high-performance cameras and, possibly, other sensors that can perform measurements, record metrics about a product or process, and compare the live data with pre-established criteria in real time. When the target for inspection meets the criteria, the user receives a signal to proceed.   AR-enabled inspection can be performed with wearable, handheld or projection (spatial) technologies and connected with tools or instruments for measuring physical properties. When connected via a network with company policies and certification systems, the results of the inspection can directly be recorded and the employee does not need to interface directly with those services and explicitly log the inspection results. A digital record of the AR-enabled inspection will also be stored and easier to retrieve, if  when and/or when requested.

Common roles of Users

Any professional who needs to receive and to follow precise instructions.

  • A professional who works in an industry that is highly regulated and the completed process needs to be inspected
  • Inspectors

Business Benefits:

The primary benefits of AR-assisted inspection are the reduction in time and errors when reviewing a task, procedure or product for compliance with policies and criteria.  With inspection use cases, the AR-enabled employee may perform some or all of a review process without experiencing downtime while awaiting the inspector’s arrival or during a manual review. Valuable inspectors time can be assigned to only those sites and procedures that are unsuitable for AR-enabled inspections.  Provided that the conditions where the inspection is performed meet criteria (e.g., excellent, balanced lighting) and the product or process does not require movement or operation, there may be also lower variability in machine vision-based inspection than with human inspectors.

Requirements

AR Hardware Requirements: Wear Ability / Comfort - Assisted

Assisted – The device weight worn on the head shall range between 125 grams and 420 grams The device shall support the addition of prescription lenses or fit over users spectacles The device shall have the ability to automatically adjust to the users Inter Pupillary Distance (IPD) to assist with ensuring eye comfort.

AR Hardware Requirements: Visual Tracking – Mobile

Mobile - The device AR object positioning accuracy MUST be within +XXmm. The device MUST be able to scan QR code (of 2 by 2 inches) from a minimum of 5 feet from a + 60 degree off-axis. Note: Optical and/or software zooming maybe utilized.  

AR Hardware Requirements: On-board Storage – Projection

Projection - The device MUST have a minimum on-board memory storage of >128GB  to support the OS, advanced perception and applications Projection - The device MUST have a minimum on-board RAM of >8GB.

AR Hardware Requirements: On-board Storage – Augmented

Augmented - The device MUST have a minimum on-board memory storage of 128GB  to support the OS, advanced perception and applications Augmented - The device MUST have a minimum on-board RAM of 8GB.

AR Hardware Requirements: Inputs / Outputs: Sensors - Mobile

The device SHOULD have an IMU with accelerometer, gyrometer and magnetometer that provides information to the software about gaze, and position. The device SHOULD have a RGB camera to enable see-what-I-see, bar-code scanning and video capture. Mobile - The device COULD have a time of flight depth sensor to measure distance and depth of the world  

AR Hardware Requirements: Inputs / Outputs: Sensors - Augmented

The device SHOULD have an IMU with accelerometer, gyrometer and magnetometer that provides information to the software about gaze, and position. The device SHOULD have a RGB camera to enable see-what-I-see, bar-code scanning and video capture. Augmented - The device MUST have time of flight depth sensor, to measure distance and depth of the world Augmented - The device MUST have two or more world cameras to enable location mapping and world reconstruction. Augmented - The device MUST have two or more eye tracking cameras to enable eye comfort and advanced perception capabilities such as a method of gaze.  

AR Hardware Requirements: Inputs / Outputs: Perception - Augmented

The device SHOULD support voice control as a method for software user interface commands. Augmented - The device COULD support eye tracking as a method of gaze, for software user interface commands. Augmented - The device COULD support gestures as a method for software user interface commands.

AR Hardware Requirements: Inputs / Outputs: Audio - Augmented

Augmented - The device COULD have multiple speakers to enable spatial audio

AR Hardware Requirements: Battery Life - Augmented Reality HMD

Under typical use, the device battery life of an Augmented Reality HMD using an AR solution SHOULD be a minimum of 8 hours on standby and 3 minimum hours usage. Under heavy use, the device battery life of an Augmented Reality HMD using an AR solution MUST be a minimum of 2 consecutive hours usage.

AR Hardware Requirements: Inputs / Outputs: Augmented

Augmented - The device MUST support a minimum resolution of 1280x960

AR Hardware Requirements: Inputs / Outputs: Assisted

Assisted - The device MUST support a minimum resolution of 854x480

AR Hardware Requirements: Field of View - Augmented

Augmented - The device MUST provide a 3D view (the images for both eyes are fully overlapping, just offset to provide the perception of 3D).

Augmented - The device MUST provide a minimum 35 degree diagonal field of vision where AR content can be displayed. Augmented - The device SHOULD provide a minimum 50 degree diagonal field of vision where AR content can be displayed.

Augmented - The device SHOULD have a variable range of operation from 40 cm to infinity, with HW support for developers to utilize from 20cm

AR Hardware Requirements: Field of View – Projection

Projection - The device MUST provide a minimum 60 degree field of vision in both directions (horizontal, vertical) where AR content can be displayed.

AR Hardware Requirements: Environmental - Augmented

Augmented -  The device SHOULD work in ambient temperatures 10ºC to 30ºC, 5 to 95% relative humidity Augmented -  The device displays SHOULD provide visible (easy to read) displays while in full ambient light conditions. Augmented - Commercial devices SHOULD work in environmental requirements documented in IP20 or greater Augmented - Industrial devices MUST work in environmental requirements documented in IP53 or greater

AR Hardware Requirements: Environmental - Assisted

Assisted - The device MUST work in ambient temperatures 0ºC to 35ºC/32ºF to 95ºF, 5 to 95% relative humidity Assisted - The device SHOULD work in environmental requirements documented in IP64 or greater

AR Hardware Requirements: Battery Life - Augmented Reality HMD

Under typical use, the device battery life of an Augmented Reality HMD using an AR solution SHOULD be a minimum of 8 hours on standby and 3 minimum hours usage. Under heavy use, the device battery life of an Augmented Reality HMD using an AR solution MUST be a minimum of 90 consecutive mins usage.

AR Hardware Requirements: Battery Life - Assisted Reality HMD

Under typical use, the device battery life of an Assisted Reality HMD using an AR solution SHOULD be a minimum of 12 hours on standby and 6 minimum hours of usage. Under heavy use, the device battery life of an Assisted Reality HMD using an AR solution MUST be a minimum of 3 consecutive hours usage.

AR Hardware Requirements: Battery Life - Mobile and/or Tablet

Under typical use, the device battery life of a Mobile and/or Tablet using an AR solution SHOULD be a minimum of 24 hours on standby and 12 minimum hours usage. Under heavy use, the device battery life of a Mobile and/or Tablet using an AR solution MUST be a minimum of 4 consecutive hours usage.

AR Hardware Requirements: Inputs / Outputs: Controller

Augmented - The device may support connection to a 6DoF controller for high precision control. (1st party)

AR Hardware Requirements: Inputs / Outputs: Mouse / Touchpad

The device COULD support connection to a Bluetooth mouse / Touchpad style device.

AR Hardware Requirements: Inputs / Outputs: Perception

The device SHOULD support voice control as a method for software user interface commands.  

AR Hardware Requirements: Inputs / Outputs: Processing

CPU (per device category) GPU (per device category) To be defined

AR Hardware Requirements: Inputs / Outputs: Sensors

The device SHOULD have an IMU with accelerometer, gyrometer and magnetometer that provides information to the software about gaze, and position. The device SHOULD have a RGB camera to enable see-what-I-see, bar-code scanning and video capture.

AR Hardware Requirements: On-board Storage - Assisted

Assisted - The device MUST have a minimum on-board memory storage of 64GB  to support the OS, advanced perception and applications Assisted - The device MUST have a minimum on-board RAM of 4GB.  

AR Hardware Requirements: Safety

Industrial devices MUST be intrinsically safe. The device SHOULD be able to be worn while wearing the required safety headgear (hard hat or bump cap). The device SHOULD meet OSHA and MSHA requirements for safety glasses. The device SHOULD have the option to meet side shield safety requirements. Electromagnetic - to be defined

AR Hardware Requirements: Visual Tracking - Augmented

Augmented - The device SHOULD perform on-board computing of spatial maps Augmented - The device SHOULD allow developers to access the computed spatial maps for application and tracking sdk development Augmented - The device AR object positioning accuracy MUST be within +5mm The device MUST be able to scan QR code (of 2 by 2 inches) from a minimum of 5 feet from a + 60 degree off-axis. Note: Optical and/or software zooming maybe utilized.

AR Hardware Requirements: Wear Ability / Comfort - Augmented

Augmented - The device weight worn on the head shall range between 260 grams and 560 grams The device shall support the addition of prescription lenses or fit over users spectacles The device shall have the ability to automatically adjust to the users Inter Pupillary Distance (IPD) to assist with ensuring eye comfort.

AR Software Requirements: AR Content Support

3D Visual Content
  • The content generation and consumption tool MUST support open 3D model formats.
  • The content generation and consumption tool SHOULD support proprietary 3D model formats.
  • The content generation and consumption tool SHOULD support animations.
2D Visual Content
  • The content generation and consumption tool MUST support open 2D formats.
  • The content generation and consumption tool SHOULD support proprietary 2D formats.
Media Content
  • The content generation and consumption tool SHOULD support open video formats.
  • The content generation and consumption tool SHOULD support open audio formats.

AR Software Requirements: AR Content: SOP

User Privileges The solution SHOULD require acknowledgment a Standard Operating Procedure (SOP)

AR Software Requirements: AR Localization

Matching the user's current environment with a previously mapped environment, along with understanding the user's current placement inside that digital environment. The device MUST support AR localization The platform SHOULD support Fast localization via localization hinting (e.g., QR Code, ArUco, or Assisted GPS) The platform SHOULD support marker-based tracking (e.g., using ArUco, AprilTag, or QR Code) The platform COULD support model-based tracking (e.g., CAD, scanned models) The platform COULD support real-time spatial tracking (e.g. SLAM)

AR Software Requirements: Integration

The solution SHOULD share data across integrated systems (i.e. LMS, MES, ERP, PLM) The solution COULD share data across users (e.g., between users during shift changes, knowledge base repositories, etc.)

AR Software Requirements: Manageability - on device

Embedded The platform SHOULD support 3rd party MDM/UEM agents Embedded The platform SHOULD support virtual private networks (VPNs) Embedded The platform MUST manage or disable device settings Embedded The device COULD support locked task mode which locks user from accessing restricted applications and device settings

AR Software Requirements: Mobile Device Management (MDM) - enterprise control

The platform MUST be able to control the permission to access third-party developers access to sensors and cameras The platform MUST allow MDM to control direct install/side load The platform SHOULD manage or disable device settings The device SHOULD support multi-user logins The platform SHOULD support over-the-air (OTA) or similar device provisioning The platform SHOULD support application management via MDM The platform COULD support remote access via MDM

AR Software Requirements: OS Features

Embedded The device software MUST support a single user accounts per device Embedded The device software SHOULD support multiple user accounts per device Embedded/Application The device software SHOULD support a web browser Embedded The device SHOULD allow third-parties developers the ability to read and write stored digital content Embedded The device MUST support file transfer Embedded The device MUST allow third-party developer access to sensors and cameras Embedded The device SHOULD support direct install/side load Embedded The device SHOULD support GDPR

AR Software Requirements: Production Deployments

Application The application SHOULD be deployable into multiple environments including on-premises (local cloud) and public cloud Application The applications MUST support General Data Protection Regulation (GDPR) requirements Application The application COULD be deployable into the Hybrid Cloud Application/Embedded The device MUST support access while off-network The platform COULD support the requirements of the Open AR Cloud Association

AR Software Requirements: Security

Application - the device SHOULD at least adhere to IEE/UL P2933 Application - the platform SHOULD support 3rd Party VPN Application - the platform SHOULD support Zero Trust configurations Application - the platform SHOULD use X509 certificates Application - the platform COULD support AES 256 device encryption Authentication - the solution SHOULD authenticate the user

AR Software Requirements: Software Developer Kits (SDKs) and Tools

The platform SHOULD support commonly available desktop software development tools The native SDK MUST enable access to device-specific functions The SDK SHOULD support common available AR deployment tools (e.g. Unity, Unreal, etc) The SDK SHOULD support cross-device development (e.g., Windows, MacOS, and Linux) The platform COULD support a cross-platform open AR Cloud SDK (https://github.com/OpenArCloud) Only Augmented - The platform SHOULD be OpenXR compliant The SDK MUST allow third-party developer access to sensors and cameras

AR Software Requirements: Supporting Use Cases: Authoring Augmented Reality Content

The authoring solution MUST support AR localization The authoring tool MUST originate content in the digital environment

AR Use Case Requirements: Maintenance

The user MUST be able to provide feedback on the step completed The solution COULD be situation-aware The solution SHOULD be location aware

Tags

All of the above Assisted Augmented Device Projection Tablet / phone 

Settings

Industries

Architecture Automotive Aviation and Aerospace Chemical Commercial Vehicle Construction and Engineering Education Emergency Response Food and Beverage Government Healthcare Industrial Equipment Logistics Manufacturing Marine Medical Metals & Mining Oil & Gas Power & Energy Smart Cities Telecommunications Transportation Utilities