Visualization

Visualization

This use case pertains to using Augmented Reality to permit employee visualization of digital assets about people, places or objects in real world context. The assets may be about the status of instruments or processes, infrastructure, or moving objects in a zone or confined space (e.g., factory), used to support acquisition of new skills or decision making. This use case can be a component of and overlaps with training, situational awareness, simulation and other use cases.

Prior to AR Adoption

Visualization is defined as having one or more of the following characteristics:

  • Involves introducing digital assets including text, graphics, models, animations or video into the users field of view.
  • Digital content of the visualization may be anchored to and registered with objects, places or people, but this feature is not required.
  • Digital content of the visualization may be real world scale, or smaller or greater than planned or actual size.
  • Users may use voice, gestures or other input methods to interact with (query or modify) the content but this feature is not required.
  • Two or more users can visualize the same digital assets at the same time.

Visualization differs from simulation in that it is not limited to models or projecting a trajectory or process in time. Without Augmented Reality, visualizations use either Virtual Reality or a 3D or 2D screen. For VR, the visualization may include a fixed or customizable environmental component.

Business Challenges AR Introduction Addresses

Visualizations can be used for understanding the benefits or limitations of different scenarios, however, when the scenarios involve new or existing objects or places, a visualization (e.g., a model of a proposed product or process) may require substantial resources for only a brief time or single use.   Without AR, the alternative to physical models is using a 2D screen. Viewing digital assets about different scenarios on a screen lacks the verisimilitude of labels on physical objects or a scale model. Designers and others who are evaluating or reviewing proposed projects, or exploring proposed modifications to existing real world places or objects, may have difficulty detecting or resolving misunderstandings that occur when visualizing proposals on a screen.   A plant manager can use visualization to weigh the pros and cons of different positions for additional machinery or ventilation systems, but if shown only on a screen, navigating the visualization to verify any conflicts is more difficult than moving around in the real world space. AR-enabled visualization places the elements in context.  Complex data sets often have a temporal axis. Visualizations of change over time is highly valuable for decision making but may not be intuitive on a computer screen and VR visualization could be costly to prepare. AR-enabled visualizations are directly produced from original data.  Visualization is also valuable when performing some tasks. For example, a live 3D visualization can assist an employee to stack boxes on a palette or load a truck with maximum stability and weight distribution.  However, if the visualization is not overlaid on the real world, the user must alternate the focus of their attention from the real world to the visualization. When not looking at the visualization the user must retain the information and transpose it to the real world. With AR, the instructions and positions of the next element is clear.

Use Case with AR

When a person or team needs to visualize an object or process and its interactions with a real world environment, digital assets in 2D or 3D can be explored or steps followed correctly in an AR experience and the real world serves as the high fidelity “background” or context for the visualization.   When used for design and review processes with the ambient lighting and other physical properties of the proposed setting, visualization of a proposal or project with AR is immersive. The AR system user has the experience as close to reality as possible and there is no need to switch focus of attention to understand the digital assets in context.  The type of AR display used by the technician depends on many factors:

  • Need for those using the visualization to have both hands free to manipulate objects
  • Room in the vicinity where the visualizations are performed for another screen pointed directly at the work space
  • Support for introducing new display devices (e.g., wearable AR, projection AR)

With AR-enabled visualizations, there is usually real time data coming from the source to the user’s experience and vice versa. Modifications proposed in the AR-enabled visualization can be traced and made in the original data file, reducing errors or misunderstandings when the visualization is no longer experienced or shared.  As in VR visualizations, the interactions and modifications of different reviewers or users can be logged by the system for future reference and/or comparisons.

Common roles of Users

  • Designers
  • Project reviewers
  • Warehouse staff
  • Cargo or shipping staff

Anyone that can benefit from seeing digital assets in the real world as a substitute for having a scale model, physical labels or documentation when performing complex data analyses, exploration, design and review.

Business Benefits:

The benefits of AR-enhanced visualization include reducing costs, by comparison to other forms of visualization, increased understanding of choices or conditions and, based on deeper understanding or interactions with the data, improved decision making. It can also be measured in terms of reduced time to complete procedures with lower (or no) errors.   In addition, there can be benefits due to lower training time and higher overall productivity of designers when they do not have to build a scale model, when reviewers of a project do not need to travel to see a project to evaluate its feasibility or even end users performing complex operations.

Requirements

AR Hardware Requirements: On-board Storage – Augmented

Augmented - The device MUST have a minimum on-board memory storage of 128GB  to support the OS, advanced perception and applications Augmented - The device MUST have a minimum on-board RAM of 8GB.

AR Hardware Requirements: Inputs / Outputs: Sensors - Augmented

The device SHOULD have an IMU with accelerometer, gyrometer and magnetometer that provides information to the software about gaze, and position. The device SHOULD have a RGB camera to enable see-what-I-see, bar-code scanning and video capture. Augmented - The device MUST have time of flight depth sensor, to measure distance and depth of the world Augmented - The device MUST have two or more world cameras to enable location mapping and world reconstruction. Augmented - The device MUST have two or more eye tracking cameras to enable eye comfort and advanced perception capabilities such as a method of gaze.  

AR Hardware Requirements: Inputs / Outputs: Perception - Augmented

The device SHOULD support voice control as a method for software user interface commands. Augmented - The device COULD support eye tracking as a method of gaze, for software user interface commands. Augmented - The device COULD support gestures as a method for software user interface commands.

AR Hardware Requirements: Inputs / Outputs: Audio - Augmented

Augmented - The device COULD have multiple speakers to enable spatial audio

AR Hardware Requirements: Inputs / Outputs: Augmented

Augmented - The device MUST support a minimum resolution of 1280x960

AR Hardware Requirements: Field of View - Augmented

Augmented - The device MUST provide a 3D view (the images for both eyes are fully overlapping, just offset to provide the perception of 3D).

Augmented - The device MUST provide a minimum 35 degree diagonal field of vision where AR content can be displayed. Augmented - The device SHOULD provide a minimum 50 degree diagonal field of vision where AR content can be displayed.

Augmented - The device SHOULD have a variable range of operation from 40 cm to infinity, with HW support for developers to utilize from 20cm

AR Hardware Requirements: Environmental - Augmented

Augmented -  The device SHOULD work in ambient temperatures 10ºC to 30ºC, 5 to 95% relative humidity Augmented -  The device displays SHOULD provide visible (easy to read) displays while in full ambient light conditions. Augmented - Commercial devices SHOULD work in environmental requirements documented in IP20 or greater Augmented - Industrial devices MUST work in environmental requirements documented in IP53 or greater

AR Hardware Requirements: Inputs / Outputs: Controller

Augmented - The device may support connection to a 6DoF controller for high precision control. (1st party)

AR Hardware Requirements: Inputs / Outputs: Mouse / Touchpad

The device COULD support connection to a Bluetooth mouse / Touchpad style device.

AR Hardware Requirements: Inputs / Outputs: Perception

The device SHOULD support voice control as a method for software user interface commands.  

AR Hardware Requirements: Inputs / Outputs: Processing

CPU (per device category) GPU (per device category) To be defined

AR Hardware Requirements: Inputs / Outputs: Sensors

The device SHOULD have an IMU with accelerometer, gyrometer and magnetometer that provides information to the software about gaze, and position. The device SHOULD have a RGB camera to enable see-what-I-see, bar-code scanning and video capture.

AR Hardware Requirements: Visual Tracking - Augmented

Augmented - The device SHOULD perform on-board computing of spatial maps Augmented - The device SHOULD allow developers to access the computed spatial maps for application and tracking sdk development Augmented - The device AR object positioning accuracy MUST be within +5mm The device MUST be able to scan QR code (of 2 by 2 inches) from a minimum of 5 feet from a + 60 degree off-axis. Note: Optical and/or software zooming maybe utilized.

AR Hardware Requirements: Wear Ability / Comfort - Augmented

Augmented - The device weight worn on the head shall range between 260 grams and 560 grams The device shall support the addition of prescription lenses or fit over users spectacles The device shall have the ability to automatically adjust to the users Inter Pupillary Distance (IPD) to assist with ensuring eye comfort.

AR Software Requirements: AR Content Support

3D Visual Content
  • The content generation and consumption tool MUST support open 3D model formats.
  • The content generation and consumption tool SHOULD support proprietary 3D model formats.
  • The content generation and consumption tool SHOULD support animations.
2D Visual Content
  • The content generation and consumption tool MUST support open 2D formats.
  • The content generation and consumption tool SHOULD support proprietary 2D formats.
Media Content
  • The content generation and consumption tool SHOULD support open video formats.
  • The content generation and consumption tool SHOULD support open audio formats.

AR Software Requirements: AR Content: SOP

User Privileges The solution SHOULD require acknowledgment a Standard Operating Procedure (SOP)

AR Software Requirements: AR Localization

Matching the user's current environment with a previously mapped environment, along with understanding the user's current placement inside that digital environment. The device MUST support AR localization The platform SHOULD support Fast localization via localization hinting (e.g., QR Code, ArUco, or Assisted GPS) The platform SHOULD support marker-based tracking (e.g., using ArUco, AprilTag, or QR Code) The platform COULD support model-based tracking (e.g., CAD, scanned models) The platform COULD support real-time spatial tracking (e.g. SLAM)

AR Software Requirements: Integration

The solution SHOULD share data across integrated systems (i.e. LMS, MES, ERP, PLM) The solution COULD share data across users (e.g., between users during shift changes, knowledge base repositories, etc.)

AR Software Requirements: Manageability - on device

Embedded The platform SHOULD support 3rd party MDM/UEM agents Embedded The platform SHOULD support virtual private networks (VPNs) Embedded The platform MUST manage or disable device settings Embedded The device COULD support locked task mode which locks user from accessing restricted applications and device settings

AR Software Requirements: Mobile Device Management (MDM) - enterprise control

The platform MUST be able to control the permission to access third-party developers access to sensors and cameras The platform MUST allow MDM to control direct install/side load The platform SHOULD manage or disable device settings The device SHOULD support multi-user logins The platform SHOULD support over-the-air (OTA) or similar device provisioning The platform SHOULD support application management via MDM The platform COULD support remote access via MDM

AR Software Requirements: OS Features

Embedded The device software MUST support a single user accounts per device Embedded The device software SHOULD support multiple user accounts per device Embedded/Application The device software SHOULD support a web browser Embedded The device SHOULD allow third-parties developers the ability to read and write stored digital content Embedded The device MUST support file transfer Embedded The device MUST allow third-party developer access to sensors and cameras Embedded The device SHOULD support direct install/side load Embedded The device SHOULD support GDPR

AR Software Requirements: Production Deployments

Application The application SHOULD be deployable into multiple environments including on-premises (local cloud) and public cloud Application The applications MUST support General Data Protection Regulation (GDPR) requirements Application The application COULD be deployable into the Hybrid Cloud Application/Embedded The device MUST support access while off-network The platform COULD support the requirements of the Open AR Cloud Association

AR Software Requirements: Security

Application - the device SHOULD at least adhere to IEE/UL P2933 Application - the platform SHOULD support 3rd Party VPN Application - the platform SHOULD support Zero Trust configurations Application - the platform SHOULD use X509 certificates Application - the platform COULD support AES 256 device encryption Authentication - the solution SHOULD authenticate the user

AR Software Requirements: Software Developer Kits (SDKs) and Tools

The platform SHOULD support commonly available desktop software development tools The native SDK MUST enable access to device-specific functions The SDK SHOULD support common available AR deployment tools (e.g. Unity, Unreal, etc) The SDK SHOULD support cross-device development (e.g., Windows, MacOS, and Linux) The platform COULD support a cross-platform open AR Cloud SDK (https://github.com/OpenArCloud) Only Augmented - The platform SHOULD be OpenXR compliant The SDK MUST allow third-party developer access to sensors and cameras

AR Software Requirements: Supporting Use Cases: Authoring Augmented Reality Content

The authoring solution MUST support AR localization The authoring tool MUST originate content in the digital environment

Tags

Augmented Device Tablet / phone 

Settings

Industries

Architecture Automotive Aviation and Aerospace Chemical Commercial Vehicle Construction and Engineering Education Emergency Response Farming Food and Beverage Government Healthcare Industrial Equipment Logistics Manufacturing Marine Medical Metals & Mining Oil & Gas Power & Energy Smart Cities Telecommunications Transportation Utilities