Over the course of 2021, Big Tech went all in on immersive collaboration and anticipation certainly grew for a game changing XR device (or two) from Apple. 2021 was also the year of the word ‘Metaverse,’ which we see as any immersive world where you might have your next team meeting or interact with brands. Let’s see what the major players in enterprise XR unveiled and acquired this year:

AUGMENTED/MIXED REALITY

MICROSOFT

Microsoft’s activities this year made it clear the company is focusing on collaborative XR and, more generally, the future of work. In March, Microsoft announced Mesh, a new platform for building multi-user XR apps or shared virtual experiences connecting people in different locations. Mesh will be brought into Microsoft Teams in 2022, allowing colleagues to meet as avatars in mixed reality.

META (FORMERLY FACEBOOK)

Facebook rebranded as Meta in 2021 in a PR move to distance the company from controversy, or as leadership claims, to convey the company’s new aim of building the metaverse. Meta also made its AR entrance with a pair of glasses developed in collaboration with Ray-Ban. The company’s 10-year plan for AR, however, is much more ambitious, as revealed in a blog post from Facebook Reality Labs. This includes developing its own operating system for AR and a mixed reality headset code named Project Cambria.

On the VR front, Meta is eyeing the enterprise market: In addition to rebooting its business tools for the consumer Oculus Quest 2, Facebook surprise launched Horizon Workrooms in August, a VR meeting app for remote collaboration, and just this month released its latest avatar system to all Unity developers. If Meta’s intentions aren’t clear enough, nearly a fifth of its employees are now working on AR/VR. (Pay attention to the competition between Meta and Microsoft for top virtual collaboration solution in 2022.)

GOOGLE and CISCO

Both Google and Cisco made moves to make existing video meeting apps more immersive: In July, Google Meet became available to all Google Workspace customers on the Glass Enterprise Edition 2. A few months later, Cisco unveiled Webex Hologram. Cisco says its photorealistic, real-time holograms – as opposed to Meta’s avatars – make virtual meetings more realistic and engaging. In other Google news, new job listings recently revealed the giant’s plans to create a new “Augmented Reality OS” for an unspecified “innovative AR device” intended to reach “billions.”

MAGIC LEAP

Magic Leap has been hard at work on its next-gen headset Magic Leap 2, which is set to debut in 2022 and already being used by select partners through an early access program. Here’s what we know about the upcoming enterprise-focused device: Magic Leap 2 is reportedly half the size and 20% lighter than its predecessor, with a doubled field of view and 3-4x the processing power. Meant to be an “all day, every day” device, additional improvements include improved color fidelity, text legibility, and image quality. Magic Leap also raised $500 million in new capital this year.

LENOVO

In 2021, Lenovo introduced its ThinkReality A3 glasses: The Industrial Edition tethers to a smartphone via USB-C for use in manufacturing, energy, and engineering, while a PC Edition is geared more towards office workers. Lenovo also certified the RealWear HMT-1 for its ThinkReality platform, and announced it will offer RealWear’s HMT-1 family of devices through its global sales network.

Notable Mentions

IRISTICK revealed its Visor Ex-01 smart glasses for hazardous areas in 2021, while Vuzix introduced Vuzix Shield, its first binocular AR smart glasses. Campfire raised $8 million, as well, to build an integrated hardware and software platform partially on the bones of failed startup Meta. In July, the company announced it’s working on holographic collaboration.

VIRTUAL REALITY

PICO

2021 was a good year for Pico, which raised $37 million and launched its next-gen headsets. The Pico Neo 3 Pro and Neo 3 Pro Eye (which includes Tobii eye tracking tech) are priced at $700 and $900 respectively, undercutting both Oculus and HTC.

HP

HP’s Reverb G2 Omnicept Edition, first announced in September 2020, became available in May. Priced at $1,250, the headset provides a wealth of data that can enhance training applications, with sensors for tracking eye movement, pupil size, facial expressions, and heart rate. (Watch HP’s Elias Stephan speak about the Omnicept Edition at EWTS 2021).

HTC

HTC launched two VR headsets in the Spring, along with pro tools for enterprise. The $1,300 HTC Vive Focus 3 is a business headset with 5k resolution, spatial audio, a 120-degree field of view, and new controllers. HTC also unveiled Vive Business, a suite of tools for applications like training, meetings, design reviews, and more.

VARJO

In June, Varjo unveiled its Reality Cloud platform, which allows you to scan a room and share it in photorealistic detail with someone remotely in near real time (essentially teleportation). Varjo Aero, a $2,000 pared down version of Varjo’s XR-3 headset for smaller businesses and wealthy enthusiasts, arrived in October; and Varjo showcased Varjo Lab Tools with which users can bring parts of the real world into VR. In addition, a VR training solution by Varjo and VRM Switzerland became the first to receive approval from the European Union Aviation Safety Agency, allowing pilots to earn credited training hours in virtual reality.

SOFTWARE

Remote Collaboration: Arvizio revealed Immerse 3D for iOS, Android, HoloLens, and Magic Leap, allowing users to interact simultaneously with the same 3D model in AR, for instance, during collaborative meetings on Microsoft Teams, Zoom, etc.

Easier AR: Scope AR lowered the bar for AR creation in enterprise with WorkLink Create, a browser-based AR platform enabling industrial professionals to author 3D content without coding.

Expanding footprint: TeamViewer acquired Upskill in 2021 and announced a partnership with Google Cloud to co-develop and market AR solutions built on Google Cloud. Assisted Order Picking is the first jointly developed solution, leveraging Google Glass Enterprise Edition 2 and TeamViewer’s Frontline solution.

PTC & Matterport: In April, PTC announced Vuforia Engine Area Targets. With support from Matterport, Leica 3D scanners, and NavVis’ indoor mobile mapping systems. Area Targets allows users to create digital twins of large spaces (think factories, malls, etc.). The integration of Matterport with PTC’s Vuforia platform can enrich enterprise applications such as wayfinding, virtual training, and real-time information retrieval.

Eye tracking: With eye tracking becoming the standard for next-gen VR headsets, WorldViz announced both Vizard 7, a platform for academic research and professional applications that integrates with over 150 VR devices, and SightLab VR, a drag-and-drop tool allowing anyone to set up and run eye tracking experiments in VR.

QUALCOMM & NVIDIA

Among other 2021 highlights, Qualcomm revealed Snapdragon Spaces XR Developer Platform, a kit to help developers build head-worn AR experiences. (As of November, Lenovo’s ThinkReality A3 glasses are the only supported hardware.) Competitor NVIDIA expanded its Omniverse platform allowing designers and reviewers to work together in real time in a virtual world. Available through a $9,000 annual subscription plan, NVIDIA hopes Omniverse will deliver the foundation for the metaverse…at least in enterprise.

____

In summary:

–       Apple glasses rumors

–       Mark Zuckerberg’s philosophical musings about the future of social and work

–       The Metaverse

But really, pay attention to the remote collaboration space in 2022, which will likely be the first mainstream metaverse application in enterprise; and get excited for upcoming headsets like Magic Leap 2 and a return to San Diego for the Augmented Enterprise Summit 2022 (formerly EWTS).

COVID-19 has continued to create global supply uncertainties, disruptions, and inflationary forces which are forcing companies of all sizes to better manage their supply chains. Combined with the ongoing growth of online shopping, attaining new productivity levels for product transportation, inventory management and order fulfillment will become a distinct competitive advantage.

Smart glasses are becoming a cost-effective tool to facilitate these objectives and an increasing number of the world’s largest firms are starting to move from trialing them to deploying them.

“Vuzix has spent a fair amount of time and resources honing this technology and we are now seeing growing market adoption of our products that are allowing companies to operate with greater productivity. We are pleased to be working with this client, which represents just one of multiple major retailers either implementing or testing our technology for logistics and warehouse usage,” said Paul Travers, President and Chief Executive Officer at Vuzix.

Human ability to imagine objects that are not physically present is limited. It is even more difficult for us to mentally place them in an existing environment. How often, for example, has it happened to you that a newly purchased piece of furniture was too large for the intended space?

In construction planning and architecture, this problem is amplified. Whereas in the case of the previously mentioned piece of furniture, only a single part has to be inserted into an existing space, in architecture we are often dealing with entire buildings in which floors, rooms and objects stand in a relationship to one another; and of course, the building itself as a whole must also fit into its surroundings. In this process, our lack of imagination can lead to mistakes with far-reaching consequences.

This is where technology helps our imagination tremendously. Augmented reality (AR) in combination with Building Information Modeling (BIM) ensures that we can “actually” see all objects and relationships.

What is BIM? What is AR?

Building Information Modeling (BIM) is to be understood as a digital method that is used throughout the life cycle of a building. In this process, all data and information related to the construction is stored and mapped in a BIM-enabled software.

Augmented reality is the computer-aided expansion of reality perception. Specifically in construction planning, the BIM models are “projected” into the real environment.

Benefits of AR in Construction Planning

AR applications for construction planning help our imagination tremendously. They support the entire decision-making process both on the side of the construction planner and on the side of the client. AR glasses can be used to better present and understand the planned building. Thus, decisions in the early planning phases can be made more easily and more correctly, which reduces planning and construction costs accordingly.

“Especially in the case of existing building conversion, it is advantageous if you can visualize the superimposition between the model and reality.”

DI Dr. Timur Uzunoglu, Managing Director convex ZT GmbH

AR Use Cases in Construction Planning at convex

At convex ZT GmbH, we use AR technology from the design phase to operation. With Holo-Light’s AR3S software, we bring BIM planning closer to clients and enable greater planning transparency. Building owners feel more involved in the planning process during our AR-assisted planning meetings and can make better decisions. We make AR inspections together with the builders directly on site. These AR inspections provide a direct impression on site in real time and help to weigh alternatives against each other. In revitalizations of existing buildings, it is often challenging to bring the new structures into a functioning harmony with the existing buildings, and AR helps very well there, too.

The solution works using sustainability targets from the United States Environmental Protection Agency (US-EPA), which aims to build a carbon-neutral future for the planet.

Citing EPA figures, ThirdEye said the COVID-19 pandemic sharply reduced global transport traffic, which was the “largest contributor to anthropogenic [US] greenhouse gas emissions at 29 [percent].”

ThirdEye’s AR/MR telepresence solutions allow companies to lower their carbon footprint by reducing the overall need for global transport, and the firm’s RemoteEye platform has cut onsite visits to allow significant cost savings, leading to a major improvement in return on investment (ROI).

Nick Cherukuri, Founder of ThirdEye, said his company’s RemoteEye platform aims to include a Carbon Footprint Score for its users to calculate the organisation’s carbon footprint with AR.

Explaining further on the benefits of AR technologies, he continued, stating,

“Not only are AR and MR teleconferencing platforms financially prudent due to traveling less, but by using this technology to share knowledge and operational workflows, there are tremendous carbon emission savings. For example, we can bring education and telehealth to underprivileged areas around the world with augmented and mixed reality”

The company’s RespondEye, which complies with the US Health Insurance Portability and Accountability Act (HIPAA), also allows doctors to tackle health problems for remote patients “anytime, anywhere.” Doctors can later assign patients and carers medical diagnoses and treatment options.

Enterprises can also benefit from the introduction of 3D digital twins to reduce inventory and other digital assets, ThirdEye said, adding doing so would reduce production emissions and costs.

The news comes as the US firm aims to expand its solutions to the Asia-Pacific with its X2 MR smart glasses and a major partnership with Go VR Immersive, a Hong Kong-based XR startup.

Tthe smart glasses would be deployed to remote workers across China, just shortly his firm inked a major partnership with Microsoft to deploy HoloLens 2 MR head-mounted displays in the Asia-Pacific region.

A few things mentioned include:

• Eye examinations can be done at a fraction of the cost
• Magic Leap’s next generation AR glasses are smaller lighter, faster
• They have a greater field view – this has doubled in their next gen device
• Vertical representation e.g. surgery digital content overlaid across the knee and look at virtual screens
• Bringing light dimmer to bring more focus to what needs to be concentrated on (again surgical use)

Answering criticism about lack of progress, Johnson argued that 4 healthcare companies are testing the devices right now and other industries are working with Magic Leap at the moment. These include:

• Health
• Defense and Public Sector
• Manufacturing
• Automotive and Transport
• Oil and Gas
• Architecture, Engineering, Construction (AEC)

You can watch the video here 

This offers a frontline connected worker platform for the integration of multiple assisted and augmented reality (SLAM) experiences into a high-performance industrial solution

RealWear Navigator™ 500 solution is the all-new head-mounted device product platform specifically designed to engage, empower and elevate the frontline worker for the next several years.

Building on the accumulated experience of the last four years, working with 5000 enterprise customers in 60 countries with solutions based on our HMT-1™ and HMT-1Z1™ platforms, this new product brings targeted innovation in all the key areas that matter most to achieving solid results at scale.

RealWear has been known for establishing and gaining major customer deployments for frontline worker solutions based on “assisted reality”.

The core concept of assisted reality is that it makes a different tradeoff than mixed reality. Assisted reality is better suited to the majority of industrial use cases where user safety is paramount.

The goals of assisted reality are to keep the user’s attention in the real world, with a direct line of sight, for the most part unoccluded by digital objects or “holograms” that require extra cognitive focus for humans to process.

Situational awareness of moving machinery, approaching forklifts or other vehicles, steam escape valves, slip and trip hazards and electrical and chemical hazards is key for RealWear’s customers. These are the same working environments that mandate specific personal protective equipment for safety glasses and goggles, to hard hats, hearing protection, heavy gloves and even respirators. Users in these situations mostly require both hands to be available for the use of tools and equipment, or to hold on to railings, ropework, etc.

In turn the user interface for assisted reality cannot rely on the availability of hands to operate handheld controllers, or to draw gestures in the air.  RealWear’s assisted reality solutions rely on voice recognition that is field proven in very high noise environments, plus the minimal use of head motion detection. The platform uses a single articulated micro-display easily adjusted to sit below the dominant eye that does not obstruct direct vision and provides the user a view similar to a 7-inch tablet screen at arm’s length.

A core concept of mixed reality has been the placement of virtual 3D digital objects overlaid on the physical world – such as 3D models or animations. This requires two stereoscopic see-through displays that are brought to a point of focus that typically is not in the same plane as the real-world object. The resulting vergence-accommodation conflict – where the greater convergence of the eyes when looking at near objects is in conflict with the focal distance, or accommodation of the eye’s lens needed to bring the digital image into focus – is a source of eyestrain, discomfort and in some cases headaches after extended use. In addition, in bright conditions, especially outdoors, mixed reality displays struggle to provide sufficient contrast with the real world and therefore they always either cut a significant amount of light from the real world using darkened glass or have to generate such a bright display that battery life is very short unless tethered with a cord to a separate battery pack. Both situations contribute to eyestrain with extended use.

However mixed reality applications do allow information to be overlaid on the real-world asset which in some use cases can provide an additional boost in productivity in identifying the item to be worked on.

So how could this tradeoff be solved?   Is it possible to tag or overlay information on the real 3D world while also maintaining safety, situational awareness, low eyestrain, hands-free use and full-shift battery life?

We’ve long believed that the answer lies in amping up the amount of “assistance” in assisted reality rather than solely focusing on the amount of reality, with power-hungry, wide field of view, super bright stereoscopic, transparent and ultra-high resolution displays. With advanced camera capabilities and computer-vision processing, key information about real-world assets can be placed on the camera view shown in the single, monocular, non-see-through (opaque) display.

Read more. 

A report by The Elec suggests that Microsoft and Samsung are working together on future augmented reality hardware. It is not disclosed whether this is for the consumer market, enterprise market, or both. All that is known is that the project is AR-related and may involve some sort of hardware Samsung will be producing (rather than Microsoft). Samsung’s investments in DigiLens, the company behind tech found in AR display devices, may further substantiate the idea that the former will be handling the physical gadgetry in its collaboration with Microsoft.

Apparently, several divisions of Samsung are involved in the project, with Samsung Display, Samsung Electro-Mechanics, and Samsung SDI are all tied in. This AR project started in the middle fo 2021 and aims to result in a commercially viable product by 2024.

Fully optimized for hands-free use, RealWear Navigator 500 is an innovative platform solution that combines hardware, software, and cloud-ready services with a rugged design that is one-third lighter and slimmer than the previous generation, making it easier for frontline workers to wear the device for their entire shift. The hardware is designed as a modular platform with an upgradeable 48 megapixel (MP) camera system, a truly hot-swappable battery, with Wi-Fi, and an optional 4G (and soon-to-be-available 5G) modem. The voice-controlled user interface includes unique noise-cancelation technology designed for high-noise environments. RealWear has more than 200 optimized partner apps supporting a variety of use cases, such as remote collaboration, guided workflow and IoT and AI data visualization.

Assisted reality [infographic available] is a non-immersive experience and has become the preferred Extended Reality (XR) solution for frontline industrial workers, especially where high situational awareness is necessary. Assisted reality experiences are closer to the physical world, compared to virtual reality (VR) and augmented reality (AR) experiences that immerse workers in the metaverse.

With RealWear Navigator 500, RealWear has again raised the bar for how assisted reality and other XR technologies are deployed at the world’s leading industrial companies. Automotive, logistics, manufacturing, food & beverage and energy companies, among others, can use RealWear Navigator 500 to deliver real-time access to online information and expertise to the world’s more than 100 million industrial frontline workers.

“With pandemic concerns continuing to press upon the global economy, how technology is enabling a ‘new way to work’ is very much in focus, particularly for industrial frontline workers,” said Andrew Chrostowski, Chairman and CEO of RealWear. “Today we’re unveiling something far bigger than a product. The RealWear Navigator 500 delivers the next generation of work with a ‘reality-first, digital-second’ enterprise solution for remote collaboration, operational efficiency, and hybrid work in safety-critical industries. Assisted reality – more so than augmented or virtual reality – is designed specifically for the frontline worker who requires both hands for the job, striking the perfect balance of keeping workers 100% present and self-aware with the ability to safely navigate industrial surroundings. After all, nobody wants to be near hazardous equipment with their head stuck into the metaverse.”

Read the rest of the full press release here. 

The new service is offered by Ford’s Technical Assistance Center (TAC), a centralized diagnostic troubleshooting team that provides support to all Ford and Lincoln dealerships’ technicians who diagnose and repair customer vehicles.  Dealer technicians can initially reach out to TAC specialists via a web-based portal or even on a phone.  With the new See What I See program, TAC specialists can now start a remote AR session using TeamViewer Frontline through a pair of onsite RealWear smart glasses to share, in real time, exactly what the repair technician is looking at.  TAC specialists can add on-screen annotations and additional documentation directly in the line of sight of the repair technicians, as well as zoom in, share their screen, record the session and even turn on flashlights remotely.

“My team diagnoses some of the most complex and complicated vehicle issues,” says Bryan Jenkins, TAC powertrain operations manager.  “I would frequently hear my team say that if they could only see what that technician is talking about, or what the technician is doing or how they’re completing a test, then they could solve the problem more accurately.  A picture is worth 1000 words, but sometimes that still wasn’t quite enough, and we needed a way to see something live and in action.  And that’s what really kicked this whole program off.”

Ford’s See What I See program is an additional layer of support that is already used by more than 400 dealers in the U.S., Mexico, South Africa, Thailand, Australia, New Zealand and the U.K.  Currently Ford is promoting the new program to its full network of 3,100 U.S. based dealers, with a positive response. “Feedback from the dealers has been really good,” says Jenkins.  “From the dealer technician perspective, they just turn on their smart glasses and accept an incoming call, then it is like my specialists are there looking over their shoulder to help resolve the problem.”

“We are very excited to add Ford to our growing list of forward-thinking customers that are leveraging AR solutions to improve business processes,” says Patty Nagle, president of TeamViewer Americas.  “The majority of workers globally do not sit in front of a desk.  Our goal is to enable those frontline workers with AR guided solutions to enable them to do their jobs better by digitalizing and streamlining processes.”

Rarely has industrial automation changed at such an exponential rate. The combination of various technology trends has propelled enterprises into Industry 4.0 so fast that Frost & Sullivan has already delivered an Industry 5.0 blueprint to guide the journey.

Edge-and-cloud integration, converged development environments, artificial intelligence (AI) and autonomous production are far more than conceptual. These technological innovations are already happening.

“This is a unique time in our industry,” explained Cyril Perducat, who shared the automation supplier’s plans for the immediate future at Automation Fair 2021 in Houston. “The future is a trajectory, a path that we are already on. When I think of Industry 4.0, which was first coined in 2011, there is certainly a lot of learning over the past 10 years of what Industry 4.0 can deliver. And COVID has accelerated many of those dimensions.”

Remote connectivity, advanced engineering with multiple digital twins, mixing physical and digital assets, and the change of human-machine interaction are driving industry along that path toward Industry 5.0.

Perducat questioned whether it’s too soon to look at Industry 5.0 when all the promise of Industry 4.0 has not yet been delivered, but he identified five changes that are attainable and impactful in Frost & Sullivan’s comparison of Industry 4.0 to Industry 5.0:

• delivery of customer experience,
• hyper customization,
• responsive and distributed supply chain,
• experience-activated (interactive) products, and
• return of manpower to factories.

“We are able to bring more capabilities to people,” said Perducat. “Human resources are scarce. By delivering systems that make the human-machine interaction more efficient, we make it more impactful while remaining safe.”

size=0 width=”100%” noshade style=’color:white’ align=center>

Rockwell Automation has identified four areas where technology can move companies along that journey:

• evolution of cloud, edge and software,
• universal control and converged integrated development environments (IDEs),
• AI native operation management, including software as a service (SaaS) and digital services, and
• autonomous systems and augmented workforce.

“We believe in control at the enterprise level,” explained Perducat. “We believe in systems with software-defined architecture and the underlying hardware. It doesn’t mean hardware is becoming obsolete. And it’s not that every piece of the system needs to be smart. The entire system, from the device to the edge and to the cloud, is smart. Edge + cloud architecture is fundamental.”

In the converged environment, control, safety and motion all come together and must work in an integrated fashion. This is especially true with the growth of robotics. “The boundaries between control and robotics are becoming more and more blurred,” said Perducat. “Safety is very fundamental in this more complex architecture. It does not work if it is not safe.”

Operations management becomes more efficient when AI is native to the architecture and is at the level of the enterprise. “A holistic view requires a lot of data and the ability to process that data,” explained Perducat. “Part of this has to be autonomous using the power of applied AI; it’s not just one more tool but is everywhere in the architecture. We can use AI on the machine to translate vibrations into data. We can think of AI in terms of process modeling. And model predictive control is evolving with AI. When you can orchestrate all the elements of the architecture, that is a system.”

FactoryTalk Analytics LogixAI is a modeling engine that enables closed-loop optimization through four steps—observe (sensor), infer (model), decide (controller) and act (actuator).

Finally, by transforming from automated systems to autonomous systems, it enables better decisions to expand human possibility.

AI can also help to simplify a new generation of design. “You can use AI to help to generate blocks of code, like individuals working together peer-to-peer, but one of them is AI, augmenting human possibility,” explained Perducat.

“We see the next step to autonomous manufacturing as an opportunity to deliver value to our customers,” he said. “The autonomous system is reimagining the fundamental principles of autonomous control systems. You don’t need to rip and replace. We have the ability to augment existing systems with new technology.”

Perducat stressed that it cannot be just technology innovation. “Technology only creates possibilities or potential values,” he explained. “It has to be accessible by users, so we have to innovate on the user experience point of view. We want to bring that to all the products, experiences and models. In a digital native world, innovation extends beyond technology and features.