Navigation

Navigation

This use case pertains to use of AR-enabled systems that display symbols or provide audio cues synchronized with the real world to direct a person or people between locations without delay or safety risks.

Prior to AR Adoption

Navigation is defined as assisting a professional who is moving on foot, with assistance of a piece of industrial equipment or a vehicle to travel safely and efficiently between two locations. Locations may be separated by vertical and horizontal distance, obstacles, secure entrances and other barriers.

In factories or warehouses, for example, the worker may need to avoid moving through zones where robots are operating, where materials they are transporting do not fit through openings, or through any space that is secure and requires credentials for entry/exit.

Often, the professional is expected to travel to a place which is unfamiliar/to which they have never been. Even if the destination is known, it’s not always safe and reliable to use past trajectories as the guide for traveling to the location. Conditions through which a professional may have passed before may be unsafe or impassible so just following a previous route is not always possible. Conversely, a past route may have taken into account conditions that have since changed and a shorter, more efficient route may be available. In construction sites, the professional may need to avoid zones that are under cranes, where there are active machines or building materials have recently been deposited.

Business Challenges AR Introduction Addresses

Getting professionals safely and quickly to their destination can be of utmost importance. Yet, navigation to a job site or to a destination can require significant effort and time. A broken pipe in the field or an unconscious victim in a building on fire represent different spatial challenges than finding a specific washer in a warehouse.

Some job sites are precisely defined and there may be maps provided to the technician. Other sites are only roughly defined or unknown and the user must explore in the vicinity of landmarks. In some use cases, such as in a warehouse, the precise orientation and elevation (in addition to 2D position/location) is important to reduce a user searching for a part or product.

Without Augmented Reality support, a user may be provided a printed document (e.g., a map) or an app on a smart phone. In either electronic or print formats, the technician must focus attention on both the real world and the map to follow turn-by-turn instructions.

In cases where a human with specific skills is urgently needed, such as in emergency response, every second counts, and the responder may need to receive other types of information (not related to navigation) while in transit. In some highly complex environments, such as buildings under construction, even a digital map on an app that is not continuously updated may not be current and reflect all conditions. Hence, the professional going to a location will need to evaluate the safety of proposed routes before following them.

Use Case with AR

When a professional receives an order to go to a job site or destination using an AR-enhanced and network-connected system, the route is developed in real time by way of artificial intelligence using all of the current data about conditions available to the algorithms. The professional then, visually or through tones or words heard, perceives turn-by-turn directions. The system also perceives the user’s position when arriving at the destination and is automatically dismissed.

The type of AR display used for navigation depends on many factors:

  • Need for use both hands (e.g., while operating a vehicle)
  • Size and weight of the device, if there are other things that need to be worn or carried to the destination
  • Connectivity to the main navigational systems containing all live data about the user’s environment.

An AR-enabled navigation system can be combined with other use cases so that the user is provided the turn-by-turn directions prior to or following completion of a task. By providing AR navigation, the professional in transit can focus more attention on gathering other information or being aware of the conditions surrounding them. The first and every subsequent step followed on the route defined by the system is either automatically detected by the system or manually dismissed by the technician.

At each step, the sounds and/or symbols providing the route are shown and registered. Upon arrival, the system automatically detects its status and can confirm the arrival of the professional at the assigned location in the management system. Alternatively, the technician can confirm arrival through voice, gesture or another interaction with the AR system.

Another possible feature of an AR-enhanced system for navigation is real time mapping of sites and facilities. Using the camera or other sensors on the AR display combined with real time positioning, an updated 3D model of the real world is produced for other users. The professional is contributing to the globally-consistent cloud-based map without dedicating any attention to environment or their destination.

Common roles of Users

Anyone moving between locations to perform tasks or achieve goals with least delay while being attentive to potential risks h least delay while being attentive to potential risks

Business Benefits:

The benefits of AR-enhanced navigation include providing real time instructions that do not interfere in any way with the user’s perception of the real world and following the route does not require diverting attention from the user’s environment. The route that is provided ensures minimum transit time, reducing down time, at the highest level of safety.   When available AR-enabled navigation increases the overall productivity of professionals, even if they have been to the destination before and regardless of the environment (indoor, outdoor, underground, etc).

Requirements

AR Hardware Requirements: Wear Ability / Comfort - Assisted

Assisted – The device weight worn on the head shall range between 125 grams and 420 grams The device shall support the addition of prescription lenses or fit over users spectacles The device shall have the ability to automatically adjust to the users Inter Pupillary Distance (IPD) to assist with ensuring eye comfort.

AR Hardware Requirements: Visual Tracking – Mobile

Mobile - The device AR object positioning accuracy MUST be within +XXmm. The device MUST be able to scan QR code (of 2 by 2 inches) from a minimum of 5 feet from a + 60 degree off-axis. Note: Optical and/or software zooming maybe utilized.  

AR Hardware Requirements: On-board Storage – Augmented

Augmented - The device MUST have a minimum on-board memory storage of 128GB  to support the OS, advanced perception and applications Augmented - The device MUST have a minimum on-board RAM of 8GB.

AR Hardware Requirements: Inputs / Outputs: Sensors - Mobile

The device SHOULD have an IMU with accelerometer, gyrometer and magnetometer that provides information to the software about gaze, and position. The device SHOULD have a RGB camera to enable see-what-I-see, bar-code scanning and video capture. Mobile - The device COULD have a time of flight depth sensor to measure distance and depth of the world  

AR Hardware Requirements: Inputs / Outputs: Sensors - Augmented

The device SHOULD have an IMU with accelerometer, gyrometer and magnetometer that provides information to the software about gaze, and position. The device SHOULD have a RGB camera to enable see-what-I-see, bar-code scanning and video capture. Augmented - The device MUST have time of flight depth sensor, to measure distance and depth of the world Augmented - The device MUST have two or more world cameras to enable location mapping and world reconstruction. Augmented - The device MUST have two or more eye tracking cameras to enable eye comfort and advanced perception capabilities such as a method of gaze.  

AR Hardware Requirements: Inputs / Outputs: Perception - Augmented

The device SHOULD support voice control as a method for software user interface commands. Augmented - The device COULD support eye tracking as a method of gaze, for software user interface commands. Augmented - The device COULD support gestures as a method for software user interface commands.

AR Hardware Requirements: Inputs / Outputs: Audio - Augmented

Augmented - The device COULD have multiple speakers to enable spatial audio

AR Hardware Requirements: Inputs / Outputs: Augmented

Augmented - The device MUST support a minimum resolution of 1280x960

AR Hardware Requirements: Inputs / Outputs: Assisted

Assisted - The device MUST support a minimum resolution of 854x480

AR Hardware Requirements: Field of View - Augmented

Augmented - The device MUST provide a 3D view (the images for both eyes are fully overlapping, just offset to provide the perception of 3D).

Augmented - The device MUST provide a minimum 35 degree diagonal field of vision where AR content can be displayed. Augmented - The device SHOULD provide a minimum 50 degree diagonal field of vision where AR content can be displayed.

Augmented - The device SHOULD have a variable range of operation from 40 cm to infinity, with HW support for developers to utilize from 20cm

AR Hardware Requirements: Environmental - Augmented

Augmented -  The device SHOULD work in ambient temperatures 10ºC to 30ºC, 5 to 95% relative humidity Augmented -  The device displays SHOULD provide visible (easy to read) displays while in full ambient light conditions. Augmented - Commercial devices SHOULD work in environmental requirements documented in IP20 or greater Augmented - Industrial devices MUST work in environmental requirements documented in IP53 or greater

AR Hardware Requirements: Environmental - Assisted

Assisted - The device MUST work in ambient temperatures 0ºC to 35ºC/32ºF to 95ºF, 5 to 95% relative humidity Assisted - The device SHOULD work in environmental requirements documented in IP64 or greater

AR Hardware Requirements: Battery Life - Assisted Reality HMD

Under typical use, the device battery life of an Assisted Reality HMD using an AR solution SHOULD be a minimum of 12 hours on standby and 6 minimum hours of usage. Under heavy use, the device battery life of an Assisted Reality HMD using an AR solution MUST be a minimum of 3 consecutive hours usage.

AR Hardware Requirements: Battery Life - Mobile and/or Tablet

Under typical use, the device battery life of a Mobile and/or Tablet using an AR solution SHOULD be a minimum of 24 hours on standby and 12 minimum hours usage. Under heavy use, the device battery life of a Mobile and/or Tablet using an AR solution MUST be a minimum of 4 consecutive hours usage.

AR Hardware Requirements: Inputs / Outputs: Controller

Augmented - The device may support connection to a 6DoF controller for high precision control. (1st party)

AR Hardware Requirements: Inputs / Outputs: Mouse / Touchpad

The device COULD support connection to a Bluetooth mouse / Touchpad style device.

AR Hardware Requirements: Inputs / Outputs: Perception

The device SHOULD support voice control as a method for software user interface commands.  

AR Hardware Requirements: Inputs / Outputs: Processing

CPU (per device category) GPU (per device category) To be defined

AR Hardware Requirements: Inputs / Outputs: Sensors

The device SHOULD have an IMU with accelerometer, gyrometer and magnetometer that provides information to the software about gaze, and position. The device SHOULD have a RGB camera to enable see-what-I-see, bar-code scanning and video capture.

AR Hardware Requirements: On-board Storage - Assisted

Assisted - The device MUST have a minimum on-board memory storage of 64GB  to support the OS, advanced perception and applications Assisted - The device MUST have a minimum on-board RAM of 4GB.  

AR Hardware Requirements: Safety

Industrial devices MUST be intrinsically safe. The device SHOULD be able to be worn while wearing the required safety headgear (hard hat or bump cap). The device SHOULD meet OSHA and MSHA requirements for safety glasses. The device SHOULD have the option to meet side shield safety requirements. Electromagnetic - to be defined

AR Hardware Requirements: Visual Tracking - Augmented

Augmented - The device SHOULD perform on-board computing of spatial maps Augmented - The device SHOULD allow developers to access the computed spatial maps for application and tracking sdk development Augmented - The device AR object positioning accuracy MUST be within +5mm The device MUST be able to scan QR code (of 2 by 2 inches) from a minimum of 5 feet from a + 60 degree off-axis. Note: Optical and/or software zooming maybe utilized.

AR Hardware Requirements: Wear Ability / Comfort - Augmented

Augmented - The device weight worn on the head shall range between 260 grams and 560 grams The device shall support the addition of prescription lenses or fit over users spectacles The device shall have the ability to automatically adjust to the users Inter Pupillary Distance (IPD) to assist with ensuring eye comfort.

AR Software Requirements: AR Content Support

3D Visual Content
  • The content generation and consumption tool MUST support open 3D model formats.
  • The content generation and consumption tool SHOULD support proprietary 3D model formats.
  • The content generation and consumption tool SHOULD support animations.
2D Visual Content
  • The content generation and consumption tool MUST support open 2D formats.
  • The content generation and consumption tool SHOULD support proprietary 2D formats.
Media Content
  • The content generation and consumption tool SHOULD support open video formats.
  • The content generation and consumption tool SHOULD support open audio formats.

AR Software Requirements: AR Content: SOP

User Privileges The solution SHOULD require acknowledgment a Standard Operating Procedure (SOP)

AR Software Requirements: AR Localization

Matching the user's current environment with a previously mapped environment, along with understanding the user's current placement inside that digital environment. The device MUST support AR localization The platform SHOULD support Fast localization via localization hinting (e.g., QR Code, ArUco, or Assisted GPS) The platform SHOULD support marker-based tracking (e.g., using ArUco, AprilTag, or QR Code) The platform COULD support model-based tracking (e.g., CAD, scanned models) The platform COULD support real-time spatial tracking (e.g. SLAM)

AR Software Requirements: Integration

The solution SHOULD share data across integrated systems (i.e. LMS, MES, ERP, PLM) The solution COULD share data across users (e.g., between users during shift changes, knowledge base repositories, etc.)

AR Software Requirements: Manageability - on device

Embedded The platform SHOULD support 3rd party MDM/UEM agents Embedded The platform SHOULD support virtual private networks (VPNs) Embedded The platform MUST manage or disable device settings Embedded The device COULD support locked task mode which locks user from accessing restricted applications and device settings

AR Software Requirements: Mobile Device Management (MDM) - enterprise control

The platform MUST be able to control the permission to access third-party developers access to sensors and cameras The platform MUST allow MDM to control direct install/side load The platform SHOULD manage or disable device settings The device SHOULD support multi-user logins The platform SHOULD support over-the-air (OTA) or similar device provisioning The platform SHOULD support application management via MDM The platform COULD support remote access via MDM

AR Software Requirements: OS Features

Embedded The device software MUST support a single user accounts per device Embedded The device software SHOULD support multiple user accounts per device Embedded/Application The device software SHOULD support a web browser Embedded The device SHOULD allow third-parties developers the ability to read and write stored digital content Embedded The device MUST support file transfer Embedded The device MUST allow third-party developer access to sensors and cameras Embedded The device SHOULD support direct install/side load Embedded The device SHOULD support GDPR

AR Software Requirements: Production Deployments

Application The application SHOULD be deployable into multiple environments including on-premises (local cloud) and public cloud Application The applications MUST support General Data Protection Regulation (GDPR) requirements Application The application COULD be deployable into the Hybrid Cloud Application/Embedded The device MUST support access while off-network The platform COULD support the requirements of the Open AR Cloud Association

AR Software Requirements: Security

Application - the device SHOULD at least adhere to IEE/UL P2933 Application - the platform SHOULD support 3rd Party VPN Application - the platform SHOULD support Zero Trust configurations Application - the platform SHOULD use X509 certificates Application - the platform COULD support AES 256 device encryption Authentication - the solution SHOULD authenticate the user

AR Software Requirements: Software Developer Kits (SDKs) and Tools

The platform SHOULD support commonly available desktop software development tools The native SDK MUST enable access to device-specific functions The SDK SHOULD support common available AR deployment tools (e.g. Unity, Unreal, etc) The SDK SHOULD support cross-device development (e.g., Windows, MacOS, and Linux) The platform COULD support a cross-platform open AR Cloud SDK (https://github.com/OpenArCloud) Only Augmented - The platform SHOULD be OpenXR compliant The SDK MUST allow third-party developer access to sensors and cameras

AR Software Requirements: Supporting Use Cases: Authoring Augmented Reality Content

The authoring solution MUST support AR localization The authoring tool MUST originate content in the digital environment

Tags

All of the above Assisted Augmented Device Projection Tablet / phone 

Settings

Diagnostics Distribution Factory layout Factory operations Field operations Field service and repair Installation and commissioning Maintenance Manufacturing processes Warehouse 

Industries

Automotive Aviation and Aerospace Chemical Commercial Vehicle Construction and Engineering Emergency Response Food and Beverage Government Industrial Equipment Logistics Manufacturing Marine Medical Metals & Mining Oil & Gas Power & Energy Smart Cities Telecommunications Transportation Utilities