Royal Dutch Shell plans to connect its frontline field workers with back office expertise through the use of a new augmented reality (AR) hard hat. The hands-free AR wearable comes with a built-in computer that’s voice-controlled and equipped with a micro-display and a camera. This makes it possible for Shell’s workers in the field to send pictures and video back to the office in real-time and receive over-the-shoulder assistance from remote operations experts when making repairs and doing work on sensitive equipment around the globe.

Shell teamed up with Honeywell to deploy these helmets from RealWear, which are the first wearable device certified to operate in hazardous conditions where explosive gases are present. The oil and gas giant has already field tested the devices and will now roll them out to 24 operational sites around the world.

“Just as laptops and mobile phones are standard for desk workers, voice command and augmented reality for wearable computers will become commonplace for field staff in our industry, driving safety and productivity,” says Michael Kaldenbach, lead for Shell Digital Realities, a center of excellence for the company that focuses on AR, virtual reality (VR), and mixed reality (MR).

The move marks another step in Shell’s journey to fold in AR, VR, and MR into operational and training environments through its Digital Realities group. Last fall the firm announced it was working with EON Reality to develop a global immersive training program that can be scaled across numerous industrial facilities. Developing an immersive training platform makes it possible for Shell to use ‘digital twins’ to recreate any training environment and put it in a module that can be offered on-demand and in numerous languages. This breaks down a lot of training barriers that enterprises like Shell face in training a global workforce operating in very exacting technical environments.

“Operational training lends itself well for virtual reality,” Kaldenbach said at the time. ““Shell is on a digital transformation journey. AR/VR is a key linking technology in that helps us visualize data and environments and make them tangible for our operational staff.”

According to estimates from analysis at Statista, the worldwide market for AR and VR is expected to grow from a projected $20.4 billion this year to $192.7 billion by 2022. According to the recent 2019 Augmented and Virtual Reality Survey Report from Perkins Coie, startup investors are bullish on the prospects of extended reality technologies. Around 90 percent of them think AR, VR, and XR technology will be as ubiquitous as mobile devices by 2025. They rank manufacturing and industrial applications like those Shell uses AR for to be the fourth most relevant environment for extended reality to take off, behind gaming, healthcare and medical devices, and education.

Meantime, experts with Accenture Labs say that extended reality technology “will be critical to the enterprise of the future.” They say that applications like Shell will act as the spear tip for greater extended reality usage across all industries in the near future.

“Indeed, industries with high-risk working environments, such as energy, manufacturing or construction, are already reaping the benefits of immersive learning,” an Accenture Labs report detailed. “Now, other industries are exploring the space.”

The Democratic Republic of the Congo (DRC) has complex and long standing humanitarian crises with about 13 million people in dire need of assistance. With less than one physician for every 10,000 people in DRC, and even fewer in conflict areas, rural health facilities are left without skilled staff and lack access to essential quality healthcare. This results in high morbidities, such as women and newborns dying during childbirth, and the rapid spread of disease epidemics due to inaccurate field diagnoses.

Iristick smart glasses take the portability of optical wear and the e-health solution of telemedicine, to connect remote medical staff with global medical expertise. The smart glasses make the eyes and ears of healthcare workers in remote areas and conflict zones available in real time to medical experts who can help diagnose, lend practical experience, or guide with their knowledge. Iristick has the potential to connect isolated medical staff with a global network of peer support and medical expertise.

Related articles:

Iristick selected as finalist for remote ehealth smartglasses (September 2018)

Iritistick AREA member profile

The International Air Transport Association (IATA) does a vital job in helping the air travel and air cargo industries meet the challenges of growing passenger and cargo demand, climate change and an evolving global economy that increasingly relies on goods being shipped quickly around the world by air.

IATA represents some 290 airlines in 120 countries and its member airlines carry 82% of the world’s air traffic. It is working aggressively to find and offer solutions that help its members meet a myriad of challenges.

At the World Cargo Symposium in Singapore last month, IATA Director General and CEO Alexandre de Juniac outlined some of the work his organization is doing to help spur and enable innovation in cargo handling – particularly as it relates to cargo handling warehouses, where goods shipped as part of e-commerce transactions are handled and dispatched to their ultimate destinations.

In talking about the need for change in warehouse facilities, he underscored the need to move more quickly in modernizing warehouse cargo operations and processes.

“Another example where urgent innovation is needed is the facilities we use. The e-commerce world is looking for fully automated high-rack warehouses, with autonomous green vehicles navigating through the facility, and employees equipped with artificial intelligence and augmented reality tools,” he said. “The average cargo warehouse today is an impressive sight. But there is a huge gap to fill.”

Brendan Sullivan, head of e-commerce and cargo operations for IATA, is charged with helping IATA members to speed up the modernization of warehouse facilities. And he’s become a big believer in using augmented reality to do it.

Last year, Sullivan and his team engaged with Atheer to help pilot new augmented reality options for IATA members – and were surprised and delighted by the results. He highlighted those results in a recent presentation at the World Cargo Symposium.

Sullivan said that IATA’s key findings were that it was able to achieve increased performance – and notably achieve a 30 percent improvement in the speed of cargo handling and a staggering 90 percent reduction in errors.

All of that means that air cargo carriers will be able to move more cargo through their warehouses, faster and with much greater accuracy. IATA’s implementation of Atheer makes effective use of the digital work instruction design and delivery capabilities of the Atheer AR Management Platform. It allows air cargo operators to have instant access to clear, consistent and unambiguous work instructions for key tasks (such as accepting cargo as ready for air carriage).

Those work instructions can be delivered directly into the field of view of warehouse cargo handlers (though the use of smart glasses, tablets or smartphones) and updated quickly and accurately via wireless connections from the mobile devices used by the warehouse.

IATA’s Brendan Sullivan reports that air cargo operators are excited by the prospect of implementing AR in their facilities. “People were really excited – because this solution is not dependent on hardware, is easy to trial and easy to implement,” he said. “That is one of the hooks for people when they hear about it.”

IATA is now working to test and on-board Atheer’s AR-based cargo handling work instructions at additional facilities.

Read the Atheer original blog post.

From RealWear’s Press Release:

“A new era of computing has arrived. Just as laptops and mobile phones are standard for desk workers, voice command and augmented reality for wearable computers will become commonplace for field staff in our industry, driving safety and productivity.”

Michael Kaldenbach, Digital Realities Lead, Shell

The RealWear HMT-1Z1 is an intrinsically safe voice-controlled device with a micro-display that shows an image to a user as if they were viewing a seven-inch screen. The head-mounted computer is the world’s first commercially available device that permits field workers in highly restricted ATEX Zone 1 C1/D1 zones to use a wearable device where potentially explosive gases are present, helping to reduce health and safety work hazards.

Shell is currently using the HMT-1Z1 for remote assistance. For instance, if equipment needs maintenance, a worker can get real-time assistance via a video call, allowing an expert to essentially see through the eyes of the onsite worker and offer over-the-shoulder assistance. In one example, an expert uses Augmented Reality to remotely “draw” on the worker’s screen which is visualized on the head mounted screen.

Shell has already field tested and deployed 40 RealWear HMT-1Z1s in multiple countries and under different conditions and is now looking to roll out the devices further.

“In close collaboration with Shell and together with RealWear’s technology, Honeywell helps to change the way field workers operate, enabling higher levels of competency, productivity and safety assurance. By addressing key customer needs, Honeywell supports industrial companies like Shell in their digital transformation journey.”

John Rudolph, President, Honeywell Process Solutions

Honeywell is the global supplier of the HMT-1Z1 and provides additional software applications, services and hands-on field knowledge.

“We are thrilled to be working with Shell and Honeywell on this historic connected field worker program. It is the people in the field who are at the heart of every company. This vote of confidence by Shell marks a turning point for the 2.7 billion deskless workers globally who increasingly require the same connectivity as those who sit behind a desk.”

Andy Lowery, Cofounder and CEO, RealWear

Shell is an early adopter of augmented and virtual reality solutions for plants and workers. The move to wearables and mixed reality is part of Shell’s broader digital transformation.

This award is part of a multi-year, follow-on agreement with a ceiling of $60 million of government funding.

“Since our founding, the opportunity and urgency to lead the charge to digitize U.S. manufacturing has only grown,” said Caralynn Nowinski Collens, CEO of UI LABS, DMDII’s parent organization. “It is essential for our national security and our prosperity that DMDII ramp up its efforts to pull manufacturers into the modern era.”

DMDII’s mission is to enable U.S. manufacturers to make every part better than the last. That requires production lines to be embedded with software and sensors that connect to the Internet. Only with this ability to send and receive data can the equipment improve itself and learn from every part produced in real time.

It is a goal far too big for any one company to solve on its own, which is why the Institute has attracted more than 300 partners, including Dow Chemical, Lockheed Martin, Rolls-Royce, Siemens, and McKinsey & Company.

Read the full press announcement from DMDII and UILabs.

The new release is designed to further empower enterprises with richer reporting and analytics, enhancements to the “See What I See” video call experience and ease of use updates that allow mobile device users to be more productive, more quickly.

The new release also expands the range of supported devices to include the recently-released Vuzix M300XL smart glasses and the Samsung Galaxy S6, S8 and Google Pixel 2XL mobile phones.

Reporting and Analytics Dashboard

A well-designed reporting and analytics dashboard is a vital information management asset for enterprises. In this release, Atheer provides a new dashboard to provide at-a-glance insight into how effectively business processes are being completed and the overall level of workforce engagement with the AR platform. Key metrics tracked by the dashboard include statistics on platform’s top users, number of calls and taskflows completed (a taskflow is a set of work instructions) – as well as the number of messages sent and the average duration of calls.

Screen Sharing for Web Users

In the course of their work, an enterprise employee in the field will often use their mobile device (smartglasses, a smartphone or a tablet) to make a video call to a remote expert or a colleague using Atheer. Typically, the scenario will involve the field employee sharing what they see and then having the remote expert annotate what the field employee sees – or send digital guidance including work instructions or annotated digital phones – to help them carry out their job.

Now, those remote experts can actually share their screens using the Atheer web console and enrich the ways in which they can help field employees. Screen sharing allows remote experts to move from being able to share specific digital assets with field employees to being able to share anything on their screen.

This new feature gives Atheer remote experts the ability to share their web-based screens during video calls with one or more web and device users. Those users can also now use annotations while sharing their screen, allowing more precise and effective collaboration.

Get More Productive, More Quickly

When using AR-enabled mobile devices, particularly smartglasses, starting applications can be tedious and time-consuming. To help enable users to be more productive, more quickly, Atheer introduces a “Kiosk Launcher” option in this release, which allows smartglasses to launch the Atheer app by default without having to navigate to it or go through any other intermediate start-up processes.

This new capability also makes access to the Atheer app more efficient for multiple users on the same device, because it allows the device to boot directly to – and stay in – the Atheer app without any user login.

Greater Device Choice and OS-specific Enhancements

The new Atheer release adds to the list of supported devices. The Atheer platform is now optimized for use on Vuzix M300XL smart glasses and additional mobile devices including the Samsung Galaxy S6, S8 and the Google Pixel 2XL.

In addition, this release includes many other enhancements including the arrival of contextualization on the Atheer iOS app – which allows users to retrieve specific content by scanning QR codes. As well, the new Android version of Atheer now includes the ability to force “over-the-air” updates to ensure users always access the most optimized version of the app based on their organization’s update settings.

Strong growth is expected to continue as global shipments climb to 68.6 million in 2023 with a compound annual growth rate (CAGR) of 66.7% over the 2019-2023 forecast period.

IDC forecasts shipments for virtual reality headsets to reach 36.7 million units in 2023 with a five-year CAGR of 46.7%.

“New headsets from brands such as Oculus, HTC, Microsoft, and others will help fuel the growth in 2019 and beyond,” said Jitesh Ubrani research manager for IDC’s Mobile Device Trackers.

“However, it’s not just new products from headset makers that will drive the AR/VR market forward. Qualcomm’s latest silicon is also expected to play a major role in enabling hardware partners and providing network connectivity for content creators.”

When it comes to the overall AR/VR headset market, roughly two-thirds of all headsets will be shipped into the commercial segment in 2023 as many AR headsets and a significant portion of VR headsets will cater to this audience. The types of industries and use cases for these deployments will vary dramatically, but key vertical use cases include everything from training and services to retail and design.

“The Augmented Reality side of the headset business has been largely enterprise-focused from the start, with a few notable exceptions, and we expect that trend to continue for the foreseeable future as most consumers in the near term will experience AR through their smartphone or tablet.”

In the context of the digital transformation of industrial production lines and cyber-physical systems, construction and manufacturing tasks are often executed in parallel.

Here, it has to be guaranteed that the physical world and its digital counterpart are in the same state.  Mobile augmented reality (AR) systems offer high potential for advanced inspection tasks as reality is superimposed digitally with deviations between supplied components and their CAD specifications visible at a glance.

The next section of the piece discusses Visometry.  The VisionLib system from Visometry GmbH combines 3D data with image processing for quality control solutions. Using model-based tracking technologies, CAD data is registered in real time to objects captured in camera images.

The outstanding feature is a reliable object recognition even in low-light and dynamic environments.

Visometry also develops stationary inspection systems for complex in-line production processes, which can be quickly and flexibly adapted to different product configurations. This makes it possible to check, measure and document not only the existence but also the orientation and position of components automatically during operation.

The machine learning techniques of common systems increasingly turn out to be inflexible and of too much effort, because the so-called teaching process must be repeated if changes to the product design or another product variant occur. Disadvantages that do not arise with the VisionLib system.

The article concludes by stating that the technology is currently used and approved in the automotive industry, but has significant application in other manufacturing industries.

PTC is a technology company that made a name for itself in the CAD and PLM markets over the past 30 years, but it’s also a leader in the emerging technologies of industrial IoT (IIoT) and augmented reality (AR). On the top floor, the Customer Experience Center (CXC) highlights PTC’s breadth of experience with demonstrations of real-life customer use cases and applications.

Tucked into its own space is the PTC Reality Lab, the forward-thinking team of engineers and researchers charged with pursuing uncharted technology. The team of five – all alumni of MIT Media Lab – are at the forefront of industrial innovation, and spend every day exploring concepts and developing technologies that will enhance people’s interactions with the physical and digital worlds.

Valentin Heun holds a Ph.D from the MIT Media Lab and has long been interested in how to foster better connections between technology and humans – and how user interface design, particularly with augmented reality, can empower workers.

The full interview is worth a read in its entirety. Here is a snapshot of what is covered:

What is the vision for PTC Reality Lab?

What are some of its special features?

What are the benefits of the co-location of the PTC Reality Lab within the Customer Experience Center (CXC)?

Interview highlights from an enterprise AR perspective:

The former Media Lab motto, “demo or die” – or nowadays, “deploy or die” – permeates our lab here at PTC. What that means is that one must make something a reality and not just innovate into the blue. When we research and develop technology, we do so with a focus on the needs, problems, and applications of industrial companies.

We’re using augmented reality, generative design, additive manufacturing, artificial intelligence, industrial IoT, robotics, digital twin, just to name a few technologies, all in the context of the physical reality. These are technologies that influence or empower at least one version of what we call “reality.”

To me, this is the essence of what PTC and the PTC Reality Lab does – to shape a seamless and intuitive user experience that is only one reality.

Vital Enterprises is a company based at the Runway Innovation Hub in San Francisco. The company was founded in 2013 with the original goal to pursue healthcare applications of the newly emerging Augmented Reality smart glasses technology.

The interview is well worth reading in full for any enterprise already using smart glasses or considering doing so in manufacturing.  First comes a high level overview of Vital Enterprises, then more specific use cases which help us to understand the evolution of smartglasses technology.

They wouldn’t dream of building their own hardware but are focused on AR software.  Better for them if customers are already using smartglasses.  Some of the brands they use are Epson, ODG, Vuzix and RealWear.  RealWear Inc, also an AREA member, according to Eldritch, has a fantastic device built for industrial applications and is 100% voice controlled.

A detailed use case is discussed of a problem presented by an industrial manufacturing enterprise customer, a multinational based in Italy.  They also detail their work with Coca Cola. The article concludes with some thoughts on the new capabilities that the Vital Enterprises team is excited about and the future trends in the field of AR smart glasses.