Simulation

This use case pertains to using Augmented Reality to simulate (within and in interaction with the real world) the insertion or repositioning of things using 3D models. The 3D models can be of weather patterns, energy flows, industrial equipment, infrastructure (such as HVAC) or moving objects in a zone or confined space (e.g., factory), complex processes requiring employee to lift or perform a process with an odd shape, including serious games (overlaps with training), simulation of packing diverse objects within volume (e.g., for shipping). Many simulation use cases overlap with visualization use cases and simulation can be used in skill development (training) use cases.

Prior to AR Adoption

Simulation is defined as having one or more of the following characteristics:

• Involves using a 3D model (or multiple 3D models) placed in a space in order to “imitate” what would happen in the real world. It’s chosen when the real system cannot be engaged, because it may not be accessible, or it may be dangerous or unacceptable to engage, or it is being designed but not yet built, or it may simply not exist.
• There is support for a user to manipulate the model(s) in real time.
• There may be multiple participants seeing and interacting with the simulation.

Without Augmented Reality, simulations use either Virtual Reality or a 3D or 2D screen. For VR, the entire world as well as the introduced object(s) of interest are created using software. There are dozens of very mature simulation systems with both hardware and software. There are also many simulation tools that rely entirely on software and “generic” hardware (e.g., a mobile device or computer with graphics acceleration).

Simulation is popular for training pilots, astronauts, emergency responders, medical professionals and many other roles. It is valuable for designers to see how their plans or mechanisms operate, and for design reviews. Simulation is also a common component of games, including but not limited to Serious Games in which users are presented with simulated (imaginary) real world scenarios that they need to solve or work through.

Business Challenges AR Introduction Addresses

Simulations are successful when users suspend their disbelief and act/learn as if they are in the real world.

To imitate the real world conditions fully in a VR system or on a screen, there must be significant investment of time and money to capture or to synthesize with software the objects in the real world and to include the real world properties and behaviors of those objects. If the simulation will be re-used by many users with the same or similar goals, there is clear return for this up-front investment.   When a simulation is unique or the problem it seeks to help users understand/visualize is only needed infrequently, the investment in modeling the real world components must be lower. The project may purchase, for simulation purposes, a commercial digital environment from a third party. Another alternative to purchasing a simulation of the real world from a third party is to generate elements in house, usually with low fidelity.

In all these cases, the copy of the world is less real than the real world itself. With Augmented Reality, the simulation is only of the object introduced into the real world. The interactivity with the model is identical to that which would be required for VR simulations.

Use Case with AR

When a company needs to simulate an object and its interactions with a real world environment, the 3D model, complete with properties and interactions, can be introduced into an AR experience and the real world serves as the high fidelity “background” or context for the simulation.

This is valuable for training scenarios in which there is a real world example available. It may also provide value to a person or team when an engineering project is being evaluated or reviewed. The model of a future (or historical) object may already be available from a design project or be scanned.

With AR, the object’s properties can mimic some features of the real world. For example, the indoor lighting (from above) or the sun, when the simulation is outdoors, may produce shadows which are included in the simulation.

The type of AR display used for simulation use cases depends on the following factors:

• Need for participant to use both hands
• Room in the vicinity where the simulations are performed for another screen pointed directly at the setting
• Support for introducing new display devices (e.g., wearable AR, projection AR)

With AR-enabled simulations, there is also usually an option to edit and to modify the properties of the model. As in VR simulations, the interactions and modifications can be logged by the system for future reference and/or comparisons.

Common roles of Users

Anyone that can benefit from the use of a model in the real world as a substitute for having the real object during training, exploration, design and review.

Business Benefits:

The benefits of AR-enhanced simulation include saving time and money necessary to synthesize an environment in which the user will learn about or explore an object’s fit or functions. Using the data captured by sensors on the AR device, and graphics rendering, the model of a future (or historical) object can have the same illuminations as it would if it were real.   In addition, there can be benefits due to users having lower training time with AR and higher overall “credibility” of the experience. If a VR cave or room is not available at a facility for fully-immersive simulations, AR eliminates the need to travel to a VR simulation facility or platform for limited time.