Augmenting The Augmented Reality For Enterprises

An article for Forbes entitled Augmenting The Augmented Reality For Enterprises by Nallan Sriraman, Global Technology Strategist of Unilever. Unlocking the Power of Data and Analytics via Cloud enabled Smart Digitization of business process.

Digital twins are nothing but the 3D digital replica of a physical thing. They have been in existence since the days of computer-aided design (CAD) became mainstream during the ’90s. However, they remained stand-alone replicas for the next 20 years until augmented reality (AR) became prominent in the gaming and entertainment industries.

As TechNewsWorld notes, AR — often referred to as mixed reality — is an immersive and “interactive experience of a real-world environment where computer-generated perceptual information enhances real-world objects.” The technology expands our physical world by adding a digital layer and generating the AR. We can view AR on many devices, such as mobile phones and iPads with a camera capturing sounding images in real time and superimposing the augmented portions to enhance the object or environment surrounding it. AR glasses incorporate the field of view and project images onto glasses providing the composite view. Digital twins with AR unite the physical and virtual worlds and provide a dynamic digital representation for us to interact with. AR is gaining momentum in higher education and monitoring hazardous environments like oil rigs remotely.

With advances in IoT sensors, specialized video processing in real time, support of machine learning (ML) and deep neural networks all rendered into slightly bigger eyewear, there is an opportunity for enterprises to leverage this technology. This article explores how data, ML and cloud can further augment and add value to enterprises leveraging AR.

First, A Few Industry Examples

  • Following through on a new product like an e-bike from early design to final product launch, including detailed design, road testing and road riding by the potential customer all in AR and offering guided repairs to customers.
  • Oil rigs floating in the Arctic being remotely monitored from Houston using IoT sensor-enabled digital twins.
  • Off-floor monitoring of a continuous process manufacturing like soaps and shampoos.
  • Guided assembly of complex components like airplane engines.
  • Remote-guided repairing of connected devices like ATMs, coffee machines and vending machines.
  • Remote roadside assistance for basic diagnosis for broken-down cars.

Leveraging The Data They Generate In The Enterprise

One of the significant differences between the previous generation of digital twins and AR-enabled ones is the amount of data they generate from the sensors. The increased number of sensors emitting frequent data provides a more immersive experience and offers vast operational data, and it can be precious to the enterprise. Therefore, we need to ensure we capture that data in a cloud-scale data lake, not throw it away after the first real-time use in AR glasses.

We can analyze these data in real time or offline and leverage it for many uses, such as scheduling preventive maintenance to replace a part’s wear and tear before it happens. In the field service examples, telemetry data can offer significant insights into how the end users operate the product and train them better. We can use the usage data from the field and feedback into the product design. The more sophisticated advanced process controls (APC) can interact with the physical twin with the same gesture controls from AR. They can also take operational feedback from ML algorithms and feed it into the physical twin (wherever possible today), like resetting the factory line to the next batch of products. It can adjust the process line to support the next product variant’s need, from can to bottle or skipping individual wrapping of teabags with just bulk pack.

What Do We Need From Technology?

Most large enterprises are already investing heavily in data lakes and analytics. They are modernizing their plants and warehouses with sophisticated IoT sensors. We can easily extend the same architecture to support enterprise AR. We need the following:

  1. APCs in places where we want to create digital twins to enable the generation of telemetry data from the components that form the physical twin.
  2. The IoT edge capabilities to gather the data and provide an execution environment to run cloud services in order to process data locally if required.
  3. A cloud environment to store the sensors’ and other physical twins’ data in the data lake. We need ML algorithms’ processing capabilities. We also need integration capabilities to connect operational IT systems and physical twins to execute business decisions as well as a secure and optimal network path between the IoT edges and cloud-hosted locations with minimal latency.
  4. The IoT hub capabilities in the chosen cloud to act as the central hub for bidirectional communication between the enterprise applications and the physical twins. It can support methods to control the physical twins via AR gestures.
  5. Real-time processing capabilities to process the data from the sensors and feed it into AR applications rendered in the web/mobile/holo lens to visualize and interact with the digital twin in order to understand what is going on in the physical twin.
  6. Mixed reality software development toolkits to build the digital twins and other mixed reality experiences to render on different screens, including AR glasses.

Some Challenges

Early this year, Science Daily reported that some basic AR (and VR) movements can contribute to muscle strain and discomfort in the shoulder and cervical spine. AR glasses also don’t yet offer alternatives to people with eye impairments like long-sightedness or astigmatism, forcing them to wear contact lenses that could incorporate safety issues due to prolonged use. Therefore, any enterprise-wide use of wearing these devices should adhere to strict time limits in order to avoid long-term health issues.

Traditionally, most networks supporting industrial operations are not resilient or secure, unlike typical IT networks. Therefore, before enterprises take a plunge in this, they should audit their networks and ensure they are highly resilient and secure.


Digital twins supplemented with AR and other ML capabilities offer numerous opportunities for aerospace, defense, automotive, transportation, energy and utilities. Equally, it is likely to change the manufacturing sector with the emergence of cooperative bots commingled with AR, making manufacturing more efficient and optimized while reducing the throughput times. However, users’ health and safety challenges and the cyberthreats are still at large.

Read the full original article by Forbes Augmenting The Augmented Reality For Enterprises

Back to News +

Share Article: