1

Solving the Skills Crisis with Augmented Reality

Where have all the workers gone?

 

The skills gap is the greatest threat to manufacturing today, second only to supply chain challenges: Nearly 45% of manufacturing leaders admit having turned down business due to lack of talent, while nearly half of frontline workers say the skills gap negatively impacts their productivity.

To understand why AR helps address the skills gap, it’s helpful to look at some of the factors that produced the gap in the first place:

Shrinking Workforce

Starting in the Eighties, a stigma around blue-collar work and the heightened importance of a college education led to fewer and fewer high school graduates joining the trades. Generational misconceptions about manufacturing work, automation and outsourcing, and the pandemic also contributed to the dwindling talent pool. Now, as Baby Boomers age out of the workforce, younger generations aren’t replacing them.

Pace of Innovation

Technology like automation, robotics, and 3D printing are slowly moving people off the factory floor into roles demanding more advanced (digital) skills. At the same time, increasingly complex products and processes are raising skill requirements in manufacturing and turning the skills gap into a moving target.

Outdated Tools

Despite talk of smart factories, old equipment and paper-based processes remain in manufacturing: 81% of frontline workers still use paper for day-to-day tasks, while 76% rely on in-person communication. In most factories, machinery older than some workers is a common sight alongside newer electronics. This complicates digitization and training and turns off younger workers.

Ineffective Training

Today’s manufacturing workers describe a technology gap and training gap. Conventional training methods like classroom courses, shadowing, hands-on training, and written manuals are costly, limited to instructors’ availability, and insufficient for keeping up with the industry’s rising learning curve. Moreover, they’re unsuitable for digital natives.

‍New Tools for the Frontline

77% of frontline workers say they don’t have the technology they need to be productive. Optimizing these workers’ productivity is the first step to blunting the impact of the skills gap. Second is to replenish the workforce with faster training and reskilling, and third is to improve the industry’s image. Augmented Reality (AR) can accomplish all three.

Fact: Augmented Reality is Narrowing the Skills Gap

Augmented Reality presents critical, contextual information, real-time insight, and remote expertise to frontline workers at the point of need, directly in the user’s line of sight on industry-leading mobile devices and hands-free headsets. Augmented work instructions, 3D products, video tutorials, schematics, IoT data, and other digital content appear on top of the work environment, directing frontline workers to do the job efficiently and accurately the first time.

Applications

Remote Assistance

What if your senior technicians were accessible to less experienced workers anytime, anywhere? Using an AR-supporting device, a junior technician can instantly share her view of a machine with a remotely located SME. The SME sees what the tech sees (live) and can guide her through the task remotely, even annotate her screen for precision. Later, a recording of the session can be repurposed as training material.

AR-powered remote support improves first-time fix rates, reduces the need for follow-up visits, and allows senior employees who might otherwise retire the opportunity to continue offering their skills. Experts can assist more workers in one day through AR than they could traveling to each worksite, effectively scaling expertise across the workforce.

Knowledge Transfer

Another way to pass on domain expertise is to have SMEs record their actions on the job, documenting specific procedures to turn into AR training material and work instructions. By having senior technicians wear smart glasses to capture tacit institutional knowledge, new employees gain a database of best practices and standardized instructions that can be superimposed in the context of the machine or task at hand for reference.

Augmented Work Instructions

Today’s manufacturing workers often work with ambiguous, outdated paper-based instructions that are difficult to follow. This wastes time and increases the likelihood of errors. AR provides on-demand access to information and turns complex processes into fool-proof visual workflows that reduce inefficiencies, increase productivity, and improve accuracy.

Humans process visual information like AR-guided work instructions better than any other type of data. Frontline workers using AR instructions don’t need to be familiar with a particular machine to service it, so manufacturers can confidently assign tasks to employees regardless of experience.

Training

When you follow AR instructions to do a new task, you’re learning that task at the same time. This is learning by doing and how younger generations learn best. AR makes it possible to safely and effectively learn on the job, in a live production environment, with significantly less supervision or downtime. Having AR as a digital mentor frees up training resources while improving time to proficiency. Off the job, AR makes training available on-demand, visually demonstrating complex processes in a way that’s easier to grasp and more engaging than e-learning and can be consumed from anywhere.

Reskilling & Recruitment

AR enhances learning for new and seasoned workers alike. More than 100 million workers will need to find different occupations by 2030, as manual and operator positions are replaced by higher-skilled, knowledge-based roles. AR-enabled upskilling programs ensure workers don’t get left behind, aid retention, make the workforce more agile, and help convince young people that manufacturing is actually a high-tech, future-looking career.

The Future of AR in Manufacturing

The benefits of augmented reality are not theoretical. Adopters are seeing greater year-over-year improvements in training time, retention of skilled workers, and hiring costs. PTC customers report significant impact like productivity, safety, and quality despite labor shortages. Frontline workers feel the value, too, in the form of increased safety and higher productivity. By driving digital transformation into the hands of workers today, AR is creating a more resilient workforce capable of closing the skills gap once and for all.




Join Boeing’s Real-World Factory Floor XR Team

Join Boeing’s Real-World Factory Floor XR Team

While some organizations’ AR/XR efforts are still limited to experiments and proofs of concept, Boeing is moving forward with fully-deployed and industrialized factory floor solutions.

As part of that effort, Boeing’s Product Systems Information Technology & Data Analytics organization is staffing up and seeking Mid-level Factor XR Software developers to join its Factory XR team.

If you’re qualified and eager to help scale up real-world XR solutions at an industry leader, go here to learn more and apply.




Meet New AREA Member ASME

AREA: For our readers who aren’t familiar with ASME, could you tell us about its mission?

Reilley: ASME is both a professional development society and a standards development organization. As a professional development society, we encourage and nurture engineering from K through 12, up through university and throughout engineers’ careers. We operate education programs, learning and development, events, and conferences, and we provide all sorts of useful content. We look to nurture and develop the engineer throughout their entire lifetime. And we’ve been a standards development organization for 140 years.

Sanna: Our work as a standards developer started during the Industrial Revolution with boilers. Nowadays, we’ve branched out into a multitude of technologies and processes, such as verification and validation, big data, additive manufacturing/3D printing, medical devices, robotics, and much more. We’re accredited as a standards developer by the American National Standards Institute. Our real core competency is finding the need that exists in global industry or in research, finding the subject matter experts and the organizations with a grounded interest in that technology or the subject matter, convening them and applying a process to make sure that all points of view are considered so we can achieve consensus. I should also point out we’ve often pushed back against overemphasis of the letter “A” in our name because we don’t have borders, per se. Our committees that write and update standards for us and find new areas for standardization are populated by people from the four corners of the Earth.

 

AREA: How did ASME’s interest in Augmented Reality come about?

Reilley: A few years ago, we identified the need to focus on what we call digital engineering. As a society of mechanical engineers, we know full well that fluids engineering and heat transfer are the cornerstones of a mechanical engineer’s degree, but with the rapid evolution of technology, engineers also need to be very well acquainted with digital engineering technologies, which include AR and VR. We recognize that, increasingly, mechanical engineers will need to work with software engineers and Augmented Reality use cases across all industries. We just wanted to make sure that we have a seat at the table as we work towards a more multidisciplinary future.

Sanna: I would add that, from the standards perspective, we have an immediate need for our standards to adapt to new disciplines and new applications. For example, we have standards and a constituency that deal with oil and gas, for example, which includes remote inspection. Even though the technology around industrial piping systems and pressurized tanks is very well established and mature, the inspection and maintenance applications are continuing to progress. Hence our standards initiatives on remote inspection, a subject which, by the way, we’ll be hosting a virtual event about on December 8 and 9. But then we also have newer activities and standards, like mobile unmanned systems and remote-control hydraulic platforms with transport capabilities.

 

AREA: In terms of AR standards development, are you still in the early stages?

Sanna: We’re just getting started in exploring how to apply AR and remote concepts to our existing activities. The bigger question is for us to determine the state of the AR ecosystem. Is there a lack of consensus that could be ameliorated with some standardization or at least getting everyone on the same page? We’re very much in a learning process and trying to get steeped in what the AREA’s constituents are doing, what they want, what they’ve achieved so far.

 

AREA: Since AR does touch a lot of different areas, as you’ve already mentioned, how does ASME work with other organizations that might also be involved in working on interoperability and standards?

Sanna: There are occasions where we set out on a path of developing a joint standard, thus we have relationships with other standards developers on a technical level, such as the American Concrete Institute and the American Nuclear Society, to name a few. We also have relationships in which representatives from groups like the Canadian Standards Association and the Japan Society of Mechanical Engineers sit on our committees. This ensures that when these organizations write something, they don’t go off in a completely different direction than us and the global industry can move forward.

Reilley: To give you another example, we’re also conducting an activity in regenerative medicine surrounding tissue-engineered medical products, or TEMPs. We are working with the Standards Coordinating Body, the SCB, which is under the auspices of the FDA and NIST, the National Institute of Standards and Technology. ASME is working on standardizing the hardware for bioprinting technologies, IEEE is working on a standard for the electrical systems for bioprinting technologies, and ASTM is working on what are called bioinks for the actual cells that are being 3D-printed. So, again, we’re making sure that our efforts are complementary in this consortium.

 

AREA: ASME has already jumped into AREA activities by hosting the recent Digital Twin Summit. But tell us more about what you hope to get out of your AREA membership and what we can expect from ASME’s participation in the AREA.

Reilley: A large part of this is an education for ASME. We really want to connect with those in the enterprise AR industry to understand the needs and the gaps and learn how ASME can make a positive impact by lending our core competencies to the effort. We also are looking to expand our network and extend the opportunity for AREA members to collaborate on ASME event programming, webinars, and other content.

Sanna: We can also help the AREA to convene subject matter experts and thought leaders to get guidance out on the street fast, knowing that it reflects the consensus of the industry. That’s something we’re very good at. Our own constituents can also benefit through our interaction. We’re looking at how we can introduce our ASME members to the AREA so they can network, discover the state of the art and the benchmarks for various applications. We’re casting a wide net and it’s going to take us a bit of time, but we want to get the right ASME people plugged into the AREA activities that really move the needle for us.

Reilley: I wanted to add that our strength at ASME is our dedicated infrastructure for developing standards. We have degreed engineers on staff and it’s their role to manage committees and volunteers to get standards through the approval process. It’s something that we pride ourselves on, and a way that we can offer the AREA real value.

 




Rockwell Automation Interview

With a heritage dating back more than a century, Rockwell Automation has long been at the forefront of industrial automation. Today, the $7 billion company offers a broad array of solutions and services that bring “The Connected Enterprise®” to life, including automation systems, integrated control systems, and factory software. We spoke with Rockwell’s Andrew Ellis shortly after the company joined the AREA as a Sponsor member.

 

AREA: Everybody knows Rockwell Automation – it’s a huge company with lots of different businesses in industrial automation. Could give us a perspective on your specific organization within Rockwell?

 

Ellis: Sure. I’m the director of our global tech consultants within a new office called the Digital Technical Office. Our tech consultants in the field help our customers align their digital transformation requirements with our capabilities. I also oversee a group of experts in different disciplines, including XR, Extended Reality. Again, our role is really working with the customers to understand their challenges and then align our portfolio to help them along on that journey.

 

AREA: Did the interest in XR come about because customers were asking about it, or was it because you looked into the future and said this is something we need to get into?

 

Ellis: When I first looked at AR several years ago, I was a bit skeptical, but it didn’t take long to change my point of view completely. Today, I don’t know how, in the near future, any customer will be able to move forward with a modern manufacturing facility without AR. It’s now similar to how it was in my early days of automation and IT/OT; there’s a lot of education going on with our customers to make them aware of the use cases, the problems AR solves, and the real benefits in an industrial landscape. For me personally, I’ve done a complete 180-degree turn around from “not sure” to “wow, you’ve got to go all-in on AR.” From our vustomer’s point of view, more and more they are asking for it, there’s no doubt. Almost on every opportunity, there are two things that come up – machine learning and Augmented Reality. How can they take advantage of it? It’s an extremely hot topic.

 

And of course, Rockwell’s interest and activities in AR accelerated considerably three years ago, when we made a $1 billion investment in PTC to partner with PTC and make it part of our InnovationSuite. Along with that came not only ThingWorx, their IoT platform, but Vuforia, their Augmented Reality platform.

 

AREA: What are the top industrial applications for AR that you’re talking to customers about?

 

Ellis: Number one is remote support. For us, that’s the application of Vuforia Chalk. The power of AR, using the camera on your phone to actually see it and then mark it up is huge, and to me it’s a no brainer. That’s a tool that needs to be in any field service person’s bag. Another is any kind of work instruction, whether simple or complex, where you want to have consistent methodology, and you want to get away from having to read 20 pages of a manual. The ramp-up time to understand how to assemble something, or for maintenance, to disassemble and repair something, is greatly shortened. That time savings is very valuable for customers.

 

A third popular use case is when you want to get a group of unskilled people up to speed quickly on a complex operation. They can use AR to walk them through the process – here’s the tools you need, here’s the safety instruction before you begin, here is the work instruction you need to follow – all manufacturers need to have this capability within their facility.

 

AREA: So the advantages of AR in all of those use cases are greater speed and efficiency, lower cost, and greater agility, right?

 

Ellis: Right. Even though we say we digitize our manuals, we’re simply making them digital as a file. The consumption still requires you to sit there and read something, whereas having it overlaid in front of you is just a totally different and better way of visualizing work instruction. The younger workers coming out of university and college are used to this. This is the world they want to live in, so we’ve got to provide the tools that are appropriate to them.

 

AREA: Have you seen the level of knowledge among customers increase a lot in recent years, or is there still a lot of education that has to go into the process?

 

Ellis: For our customers, there’s been a huge increase in awareness of what AR is and how it’s used. They’re now more interested to know what other customers have done. Nobody wants to be that first adopter. They want to be somewhere near the middle, and not the laggards. I think we’re still on that upward curve. It definitely hasn’t become standard in every processing plant or manufacturing facility, but there’s lots of research going on regarding how to apply it.

 

They want to know what other companies are getting for a return on investment. How did they deploy? What kind of infrastructure did they need? It is something that you need to have a solid wireless infrastructure available in your facility if you plan to use it for certain things, or a 5G network, whatever you want to use, but it makes the infrastructure really a prerequisite.

 

AREA: When you’re talking to prospects, and they say give me your best case study that you can share publicly, what do you tell them?

 

Ellis: Because we’re leveraging PTC product, a lot of our case studies are from PTC, because they’ve been doing this for a lot longer than us. But we also talk about our own internal case studies, because we’re a manufacturer as well, we’ve got 20-plus manufacturing facilities globally, and using AR with our equipment makes a lot of sense. Currently we’re working on an application where you can use AR to look at our MCC motor control centers, or our variable speed drives, or variable frequency drives, and start to get IoT data out of them. And not only the data, but start to get the predictions, obviously the predictions are coming from somewhere else, but now they’re showing up in AR.

 

Or I can use it as virtual safety boundaries. If I’m using the HoloLens, and I’ve got spatial awareness, as I move into an area that may be considered dangerous, it’s fenced off virtually. We’re seeing a lot of benefit applying AR experiences to our own products to help our customers, as well as in our own manufacturing facilities when we’re doing virtual commissioning of equipment. So we can use Vuforia Chalk to look at an assembly and share it with a customer in the markup. That kind of virtual commissioning has been fantastic. So we’ve had a number of use cases where we’re using it ourselves in our own facilities.

 

AREA: Is there a vision for AR at Rockwell Automation, or is it considered just another tool in the toolbox you’re offering customers?

 

Ellis: I think it’s becoming part of our vision, a fundamental pillar in what we will provide to our customers. We can see a day when someone puts on either a 2D or 3D device, looks at their equipment or facility, and starts to visualize where the trouble spots are, what product is running through the line at this point in time, who is on the line working on it, what are the orders that are jammed up. You’ll be able to really visualize it so you don’t have to go into a traditional control room or open up your laptop to take a look at it.

 

We haven’t tapped all of it, but there’s so much interest in using AR in almost all of our products, from hardware in maintenance and safety, to assembly, to how to use, to troubleshooting, and then on the IoT side, consuming IoT data, or work instructions. It’s used everywhere, so I think it’s going to be a standard platform that you’ll see from Rockwell from now on.

 

AREA: Finally, could you tell us your reasons for joining the AREA?

 

Ellis: Our customers – whether it’s manufacturers or processing facilities – get a lot of their information from independent organizations like the AREA that are solely focused on sharing best practices standards and information gleaned from other users, and making it available to the wider ecosystem. That’s what the AREA does, and we wanted to be part of it – to contribute to it, sharing what we’re seeing as a manufacturer and what our customers are seeing, but also to gain access to the information the AREA is putting together. Your calculators are fantastic. We can point our customers to AREA literature that’s vendor-neutral – much like when I’m buying something, I want to look at Consumer Reports or another source that has no vendor bias. When you have an organization like the AREA, where you pull in vendors and users, and create material that we can consume, but more importantly share with our customers – to me that’s invaluable.




Interview with EXO Insights

New AREA member EXO Insights has carved a successful niche in the enterprise AR market as a source of biometric VR/AR solutions that transform and build on existing industrial safety and training standards to move organizations past traditional training methods. Based in Waterloo, Ontario, EXO Insights’ suite of sensors, integrated with VR/AR hardware, collects precise behavioral data generated during sector-specific designed simulations.

The EXO Analytics engine then consolidates trainees’ knowledge and behavioral data to provide organizations with actionable insights to improve industrial safety and productivity. We spoke to the company’s CEO, Fernando Muniz-Simas to learn more about EXO Insights and its unique AR-based solutions.

AREA: Welcome to the AREA, Fernando. Tell us how EXO Insights started and how you got to where you are today.

Muniz-Simas: The company started unofficially about 15 years ago, pre-Augmented Reality. We began in the Marketing area, doing research for big food conglomerates like Nestlé and Unilever. We were leveraging Virtual Reality as a way of capturing behavioral information for market research through eyetracking. We created simulated supermarket aisles and asked people to reenact their shopping trips. It was valuable information to our clients because it was unbiased; we simply measured behavior.

While the research was effective, it was very costly. So we began looking at applying our ideas to industrial training applications in the mining industry and our business took off. We established our company here in Canada in 2016 and started working in the energy and defense sectors, always in a mission-critical, high-risk environment.

AREA: Are most of your customers in Canada?

Muniz-Simas: Yes. We also have one customer in Europe and we are working to expand our business across North America.

AREA: Is your strategy to stay in the training space?

Muniz-Simas: Yes. We are also strongly focused on research. We’re very close to an important research university, the University of Waterloo, and this whole area is a great center of innovation and entrepreneurship. There are two components to our work: using sensors that measure behavior; and making that information meaningful. We could be measuring heart rate and stress, for example, but is that information meaningful to the task at hand? Research is fundamental for us.

AREA: Who’s a typical EXO Insights customer and what are they doing with your solutions?

Muniz-Simas: A good example is nuclear power generation. They are tremendously focused on safety, for a very good reason. It’s a very good match for our simulation capabilities, because we’re not really focused on training per se, but on the finer points of how behavior affects the outcomes of your training or how you perform your work. There are two ways to improve skills: classroom training and human performance – how you improve the way you do your work. It’s not necessarily related to specific skills. For example, the nuclear industry trains some of its operators in flight simulators. It’s as advanced as you get. It’s high-risk, mission-critical work.

EXO Insights helps them improve human performance by delivers training that is as close to reality as possible. Leveraging state-of-the-art VR, AR, and biometric technologies, we create a safe and hyper-real training experience that allows workers to rehearse critical tasks and put their learning to the test in a safe, simulated workplace. Simulation allows workers and organizations to safely fail, so they can learn from those failures and ensure they are not repeated in real life.

AREA: Tell us why you joined the AREA and what you hope to get out of your membership.

Muniz-Simas: Since coming to Canada four years ago, we’ve essentially become a startup again, so we’re looking for ways to learn how things are done here. I’ve always had the AREA close to my heart. The organization has always had open arms and has freely shared some really great information – for example, the ROI Calculator. There’s a lot of great thinking behind that. We saw a lot of value in the AREA and a lot of correctly-focused initiatives that will strengthen the AR ecosystem. It’s a very solid organization with some very important members. It’s something we felt we had to participate in and contribute to.




Matt Fleckenstein on Mixed Reality and New Microsoft Mesh

As a member of the AR ecosystem, you’re familiar with Microsoft HoloLens, of course. But are you aware of Microsoft’s AR strategy beyond its successful head-mounted display? To get answers, we spoke to Matt Fleckenstein, Head of Strategy, Commercial and Consumer Mixed Reality for the IT giant, which recently joined the AREA as a Sponsor member. A tech veteran with a rich and varied background, Matt gave us a fresh look at the future of AR/VR/MR at Microsoft.

 

AREA: Matt, welcome to the AREA! You’ve been in your new position at Microsoft for a few months now. Tell us about your role and the scope of your work.

Fleckenstein: I got recruited back to Microsoft two years ago, and I’ve spent most of my time driving our Mixed Reality business. I was on the product marketing team for the majority of that time, where I ran our go-to-market for our Mixed Reality business, and more recently I moved over to our engineering team to drive our Mixed Reality strategy and road map for our products and services.. We’re starting to look at what, if anything, we’re going to do beyond the VR stuff we have in market today in the consumer world. The commercial part is probably more apropos to what we’re talking about here.

AREA: You’re looking at both the commercial and consumer sides of things. Is there a lot of overlap between the two or are they pretty distinct?

Fleckenstein: It’s a great question. They’re fairly distinct today, but my point of view is that they’ll look more similar down the road. What I mean by that is, when you think about the world we’re in today, where we’re all forced to interact in much more remote manner. And whether I’m interacting with other people for work purposes or in my personal life, over time those things will look more and more alike.

A good example of that is our recent announcement of Microsoft Mesh. Microsoft Mesh is an enabling platform to usher in the era of collaborative computing, enabling users to work, play, and learn together remotely, as if they were there in person. Using any MR-enabled device (such as HoloLens, Quest, or AR-enabled phones), users can holoport themselves or their avatars into any location to enable rich, immersive shared experiences. This has broad applicability across both commercial and consumer businesses from hosting virtual conferences to remote meetings to rock concerts. When you think about it in that context, even though the content is different, the ability to connect with other humans and collaborate is a human need across our work and personal lives.

AREA: I would imagine, too, that you’re going to learn different things from the commercial and consumer sides that could be transferable between the two.

Fleckenstein: Yeah, I think that’s true. As the folks in the AREA well know, a lot of the use cases for enterprise Augmented, Mixed, and Virtual Reality today are in training. And some of that training is teaching people hard skills – for example, how to assemble a Rolls-Royce engine for an Airbus airplane. But others are more about teaching soft skills, whether it’s customer service skills or sales training and skills training. When you start to get into that world, it’s all about human connection and emotion, as well. So, while I do think they are different today, they will also definitely feed each other. There is more overlap probably than what may be apparent at first blush.

AREA: Can you give us a sense of the scope of Microsoft’s Mixed Reality strategy?

Fleckenstein: The biggest challenge I had when I was on the marketing side was overcoming the perception that we were about HoloLens and that was it. Certainly, that’s a big part of what we do. But we’ve tried to holistically build out a full Mixed Reality stack so that you can have all the tools and services running in Azure that you need in conjunction with 3D content platforms like Unity and Unreal. So, that you can build the applications and manage the applications that you need in the enterprise space – whether that’s running on a HoloLens device or a mobile device. You know, we’ve built a rich set of mixed reality Azure services and support that development with deep integration with those 3D content platforms. So, we’ve got kind of that PaaS layer that really can be helpful.

We also have our first-party SaaS applications. We have two applications in our Microsoft Dynamics 365 group: Remote Assist and Guides. Remote Assist is all about the use case for bringing in a remote expert to help you with whatever you’re doing. That could be a field service agent who’s out trying to repair a complex piece of machinery and needs an additional set of eyes and ears and hands to help do what they’re doing. Or it could be a medical professional who wants to patch in an expert to get a second opinion on something. Guides is all about the step-by-step guidance to help you perform a task. Go back to that Rolls Royce engine for an airplane, right? Teaching a new employee how to assemble or how to repair one of those – leveraging the power of overlaying 3D holographic images on top of what you’re doing to walk you through exactly what you need to do is one of the use cases that we see today. So, we’ve got our own first-party set of applications that run on HoloLens but also run on other endpoints like mobile devices and tablets.

And then we’ve got Microsoft Mesh and a variety of mixed reality PaaS services that have enabled a robust ecosystem of about 175 different enterprise ISV’s who have built applications to solve different use cases in different industries, some leveraging HoloLens and some leveraging other endpoints. The industries served include health care, manufacturing, architecture, construction, and engineering.

Then of course we’ve got the AltspaceVR service where people can gather in VR and do everything from rock concerts to conferences to virtual meetings. It’s a pretty robust stack when you look at what we’re kind of trying to build out, end to end. Ultimately, our focus is to leverage the power of Augmented, Virtual, and Mixed Reality to enhance the next generation of experiences across consumer and commercial worlds.

AREA: What’s at the top of your to-do list for 2021?

Fleckenstein: We’re doubling down on how we can, at scale, light up the next generation of collaboration as evidenced by our launch of Microsoft Mesh. A lot of the work that’s been done in enterprises today, a lot of the success that folks have had with Mixed Reality, with HoloLens, has been helping individuals complete a job. I think the opportunity exists for much more. This dialogue we’re having remotely through a 2D medium would be so much richer if you and I were sitting down having a cup of coffee in front of the fireplace having a chat. So our focus is on how we light up the next generation of collaboration, whether it’s me talking to my parents back in Punxsutawney, PA to get an update on whether Phil saw his shadow or not, or having a work meeting with a bunch of people. How can we best transcend time and space and create that next generation of collaboration?

AREA: What do you see as the most important barriers that need to come down to open up all these opportunities?

Fleckenstein: There’s a whole set of practical obstacles in the enterprise that we need to keep working through. Those include everything from making it easier for IT departments to manage a suite of different devices to enable Mixed Reality experiences in their enterprises. That’s everything from security to device management to dealing with authentication and identity. There’s more work to do to make all of that easier so that we can move from companies deploying dozens or hundreds of these devices to thousands to 10,000’s of these devices.

There are certainly companies who are leading the way on that, like Toyota, for example, which is putting HoloLens devices in every one of their dealerships in their service centers so that they can offer next-generation customer experiences. That’s going to happen in the tens of thousands, but that’s rare still today. So, the main obstacles are the physical IT deployment and rollout tasks that need more maturity.

AREA: Right. The less glamorous stuff.

Fleckenstein: Yeah, but it’s absolutely necessary. We saw it with mobile devices when people started bringing their iPhones to work. You have to do that foundational work if you want to operate it at scale. The other key is change management. New technologies are always tough to introduce. How do you bring along the people and the processes to support the evolution that’s happening? How do we manage the transition from mobile devices and computer screens to head-mounted displays and ultimately glasses?

The expanded adoption of Mixed Reality will also require us to defy physics by making the devices smaller and more lightweight while making the displays and the field of view better and bigger and improving battery life all at the same time. I do think smaller form factors that lower costs are really going to be imperative for us to really unlock the full potential of the technology. We just need to keep making the products lighter and better and bringing down that cost.

AREA: What does Microsoft expect to get out of being a member of the AREA?

Fleckenstein: One of the things that was most attractive, and one of the reasons why we joined, was to get access to a lot of the research and tools the AREA has put together. There’s just nothing else like it out there – whether it’s an incredibly robust worksheet to help you build out the business case for your organization, or the research the AREA has conducted to understand what people are really doing with the technology. What are the challenges that they’re facing? What are the requirements that aren’t being met? There’s tons of research out there about this space in general, but it’s fairly high level and it’s fairly broad. One of the biggest things the AREA has going for it is the rigor and depth of the research. It’s been fantastic.




The AREA on YouTube – why not Subscribe?

We all like to consume our content in different ways. For many of us, listening and watching can be easier and more digestible than reading. That’s why we’re pleased to offer all of the AREA’s public webinars and thought leader interviews on our YouTube channel. You’ll find years worth of great content devoted to Augmented Reality in the Enterprise there. 

Better yet, why not subscribe to keep up to date on everything connected with AR for the Enterprise?

 






Stereoscape brings AR to B2B segment in Finland

Stereoscape, a new AREA member, was founded in 2009 in Helsinki, Finland by a team of creative and technology experts specializing in stereoscopic 3D for films. Today the company is focused on using interactive 3D and XR technologies to enable people to work smarter and learn faster. We spoke with the Stereoscape team recently to learn more.

AREA: Tell us about how Stereoscape got into AR.

Stereoscape: We started 10 years ago with stereoscopic 3D conversions and live action S3D shoots for film, fashion and advertising clients, after which we moved to 3D holographic solutions for marketing and events. Over time, our focus turned to working with immersive and interactive technologies, and we now have five years’ experience in XR-based projects for enterprise clients. Throughout our 10-year journey we’ve been passionate about how the creative application of new technologies can deepen people’s experiences, whether at work or leisure.

AREA: Are you focusing primarily on the European market?

Stereoscape: At the moment, we are mainly working with large Finnish multinationals in the industrial B2B segment. Some of our clients build mechanical engineering products, while others, like Nokia, are providing technology for 5G networks and Industry 4.0. Our key clients are based in Finland and the Nordics, with our solutions often deployed internationally.

AREA: How would you describe the state of AR adoption in Finland?

Stereoscape: Finland is still very much at the emerging stages of adopting AR, with some great success stories in validated solutions, while others are still learning about the potential for AR as a driver of business value. Because Finland has a strong tech infrastructure and tech startup culture, there is generally a good appreciation for technology-driven innovation, and there’s a nice AR ecosystem forming locally.

AREA: What would you say distinguishes Stereoscape from its competitors?

Stereoscape: With the rapid pace of innovation in XR, we remain technology-agnostic. We are focused on the user experience – putting technology in the service of people. We are passionate about making people’s working lives easier by presenting information in an engaging, interactive and effective way with the help of XR. Our expert team has accumulated a wealth of experience in the creative applications of new technologies. We’ve also invested in the productization of our solutions and services to make it easier for our enterprise clients to adopt these new technologies.

AREA: On your website, you talk about “Smart Product Communication” as being the underlying theme of your work. What do you mean by that?

Stereoscape: We first coined the concept some five years ago to depict the new ways of communicating complex product information made possible by interactive 3D and XR – moving from static to dynamic, from flat to spatial, from consumption to exploration, from explanation to experience. We were then mainly working with sales and marketing solutions and have since extended our offering to other parts of the enterprise value chain. But helping people comprehend complex information is still at the heart of everything we do, whether in sales, learning or maintenance. With AR, and other XR, we have so many exciting tools available to make digital information more alive, accessible, responsive and enjoyable. I think our slogan – Experience information – nicely captures our mission of enabling people to experience information in entirely new ways.

AREA: Your fellow AREA member RE’FLEKT is one of your partners. Do you do a lot of work in collaboration with partners?

Stereoscape: We work with a broad international network of technology and other expert partners on all our projects, including developers and other subcontractors. In-house, we focus on client consultation and the design of the solution, then collaborate closely with our partner network on the implementation phase to create an end-to-end solution.

AREA: What do you hope to gain by joining AREA?

Stereoscape: We are keen to connect with other companies and our peers who are passionate about AR to exchange experiences, learn, and collaborate. AR, and XR in general, requires a wide range of experts to join forces to bring about the digital transformation made possible by new technologies.

AREA: What can we look forward to from Stereoscape in the next couple of years?

Stereoscape: Since Covid-19 broke out, we have been busy creating new solutions for online use, to help companies engage with their customers or onboard and train employees despite the meeting and travel restrictions. Looking beyond the current crisis, we firmly believe that the pace of AR innovation and adoption will only keep accelerating. We are excited by the opportunities this will bring to our clients and all those working within the field of AR. 




AREA Safety Committee Recommendations for Cleaning AR Headsets Under COVID

Whether or not you agree with the old saying that “cleanliness is next to godliness,” in the COVID-19 age, cleanliness is paramount when it comes to preventing infection. The U.S. National Institutes of Health report that SARS-CoV-2, the virus that causes COVID-19, can be detected in aerosols for up to three hours and on plastic and stainless steel surfaces for up to three days.

That’s why it’s so vital to regularly – and thoroughly – clean all AR devices, particularly for organizations where headsets and handheld devices are shared among multiple users. We consulted with the AREA Safety Committee to learn how best to handle headset cleaning.

While no official industry best practices have been established for coronavirus cleaning (and there are no guarantees that your cleaning regimen will prevent transmission of the disease), the AREA would like to share these recommendations. They begin with separate cleaning materials and processes for each AR headset component, as follows:

Optics 

Lenses and cameras can be damaged by abrasive cloths or cleaners. The Safety Committee’s advice is to unplug the camera from its power source, then wipe clean with a microfiber cloth. Lenses get the same treatment, with the microfiber cloth applied in a circular motion, beginning at the center of the lens and moving outward. 

Cushions 

To clean headset cushions, first remove the cushion from the headset. Take a soft cloth dampened with warm (40°C/104°F) water, gently wipe, and allow to dry at room temperature. 

Headset Plastics 

Hard surfaces, such as headset plastics, are where the coronavirus is likeliest to survive. The Committee recommends isolating the component, wiping with a soft cloth or cotton swab, then thoroughly wiping it down with a non-abrasive antibacterial wipe. 

Headset Electronics 

The preferred way to clean a headset’s electrical parts is by isolating the component and wiping thoroughly with a dry soft cloth or cotton swab.

 

In addition, the Safety Committee recommends avoiding exposure of devices to UV rays or direct sunlight, reducing sharing of devices when possible, and using personal or disposable covers for high-contact areas, such as brow pads.

Finally, please keep in mind that public health officials are still learning about the coronavirus and any guidance or recommendations may be subject to change. Please consult your jurisdiction’s infectious disease experts for the latest information.

The AREA Safety Committee is dedicated to identifying and prioritizing AR safety risks and recommending steps that enterprises can take to manage and reduce those risks. Visit here to learn more about the Committee, its work, and how to join.




BSI’s Tim McGarr on Enterprise AR and Standards Development

 

AREA: Where do things currently stand in terms of standards development for AR?

McGarr: There are a few things that are going on in international standards. For example, there are already standards for spatial recognition, amongst other areas. But the reason I’ve been focused on getting new standards going is that some of the barriers to AR becoming mass-market relate to aspects of the technology that are not standardized.

That’s where I see BSI coming in. As the UK national standards body feeding into European and international standards, we span a broad spectrum of standards, from door locks to ethics to cybersecurity and everything in between. For VR and AR to become a mass-market proposition, they will have to interact with the way many industry sectors already successfully work, which is generally based on longstanding areas of standardization. Such standardization is in place to deal with the challenges mature sectors have dealt with in the past. For this reason, the new standards work is focused primarily on ensuring health and safety are taken into account.

Among many industry sectors, such as construction and manufacturing, there are commonly accepted procedures and equipment to ensure health and safety. So even if AR technology delivers increased productivity, if it doesn’t align with existing health and safety procedures or equipment, it’s not going to be accepted or widely used.

There are several aspects to consider; first, ensuring that the technology itself doesn’t introduce new concerns – for example, if a person on a manufacturing floor using VR or AR stops using the technology and feels dizzy in a potentially dangerous environment. Second, the technology has to be interlinked with other standardized personal protective equipment (PPE) – such as protective gloves or hardhats. The technology has to work within those parameters which are generally supported by standardization. Then there are areas of standards that are likely to follow – for example, ensuring that a device is robust enough to work within an environment like a building site or ensuring appropriate cybersecurity and privacy management.

All our work is about taking what is a great technology proven to make organizations more efficient and get it to a place where it is accepted by most mainstream organizations.

AREA: Does BSI begin by identifying which issues are the highest priorities?

McGarr: We started by commissioning independent research to determine where standards broadly were needed. Using that as a starting point, we’ve talked with a lot of people in the market to determine what the main priorities are. So we’ll work on the main ones first and then build further out. In AR, the highest priorities are safety, which includes setup and immersion time, linking up with PPE, and cleanliness, since we have people sharing devices. Cleanliness of devices has been especially important in healthcare, but it has taken on greater importance in all sectors during the pandemic.

AREA: How does that process of industry outreach work?

McGarr: A lot of it is finding and reaching out to the right people. We think about who the main stakeholders are. That can include people from the academic world, government, trade bodies, consumer groups, as well as companies of all sizes. We work to get the right stakeholders around the table and give them the opportunity to represent their views. Part of the reason that we are AREA members is about reaching out to those people, both from the UK and the AREA members from other countries who can feed in locally.

AREA: Tell us about your recent AR and VR standards development workshop.

McGarr: We’ve spent a lot of time determining how best to proceed to get standards that can grow the AR market. International standards committees work in various ways. You can develop a new committee or start a working group within an existing committee. After lots of discussion we have recently had agreed the best approach is in forming a new “Working Group” (WG11) within the ISO/IEC SC24 committee.  This Working Group will be developing standards to build out areas of standards to aid the AR market to grow with standards specific to AR/VR such as Health & Safety, Personal Protective Equipment, hygiene, diversity/equality/inclusion, robustness, content capture/processing/postproduction, and cybersecurity/privacy/online harms. The initial standards proposals to start this work are being voted on and can hopefully start very soon.

AREA: When can we expect to see the fruits of this AR/VR standards development work?

McGarr: We hope to get work started in a few months’ time. The international standards will be there in approximately two to four years. If there is a desire to do things more quickly, there are ways to do them more quickly. It’s all about bringing stakeholders together, bringing in a wide variety of views, building consensus, and establishing best practices.

AREA: If people reading this want to get involved or get more information about your work, what should they do?

McGarr: I’m happy to be the main point of contact. People can reach me at [email protected]. We’ll also be feeding back into the various AREA groups.