Employees whose jobs affect every aspect of one’s trip, including aircraft maintenance workers and flight crew can make use of wearable XR technologies to ensure the end goal: A safe and satisfied traveler. Find out how XR might be used on the ground and in the air when you go on your next business trip or vacation.

The article goes on to walk the reader through a variety of applications in the aviation industry:

On the Ground: AR for Assembly

Both Airbus and Boeing employ augmented reality (AR) glasses in the aircraft assembly process. Airbus workers follow plans directly in their field of view, superimposed on the plane’s interior during cabin installation. They use the same solution to check the accuracy and quality of their work (image recognition technology and artificial intelligence at work); while Boeing employees use smart glasses to view a heads-up, hands-free roadmap for wire harness assembly over their real-world view. In each case, AR functions to form a stronger connection for the user between textual or diagrammatic instructions and the real working environment.

Other areas of wearable AR and VR in aviation and aerospace discussed include VR for Training, AR for Guidance in the air and XR in Flight Service.  This comprehensive and digestible article is a great resource for those interested in how wearable technology is being used in aviation.

Augmented Reality is gaining more and more in importance in industry: “We are developing software that helps us build and maintain aircraft tanks. It aims to increase the flexibility of the employees, accelerate the workflow, and connect and optimize processes,” says Christian Tesch from the Institute of Anthropomatics and Robotics at KIT, Chair of Intelligent Sensor-Actuator-Systems (ISAS, headed by Prof. Uwe D. Hanebeck).

Many commercial aircraft are at first not equipped for long flights and their fuel tanks are too small. So that they can still travel long distances, additional tanks are required which have to be maintained on a regular basis. To do that, so far engineers have had to climb into the tanks through a small opening. Often, however, they need both hands for assembling components, and new workers in particular also need instructions at the same time.

The AR glasses – at the moment we are using the HoloLens from Microsoft – display the work to be done in the engineers’ field of vision, who then have their hands free to install or repair components,” says Tesch. The glasses are equipped with cameras. Users scan special markers on the tank in advance using the cameras, which communicate the exact location and the size of the tank to the glasses. A transparent 3-D computer model from the inside of the tank is then projected onto the “real” tank; this means that engineers can also look into the closed tank from outside, understand the structure in detail, and get step-by-step instructions on how to install a pipe, for instance.

What’s more, with the help of markings on the ground, the glasses show where to find the required components in the warehouse, the location of which the glasses also recognize thanks to markers that the system has already learned. “We connect the actual work on the tank with locating objects which helps us create an overall concept,” says Tesch.

Full article here.

Augmented reality is one of the fastest growing segments of the technology market. AR superimposes digital information onto a person’s view of the physical world. For manufacturers, augmented reality can enhance all functions within the product lifecycle, including engineering, manufacturing, marketing, sales, operations, and service. The transformative nature of AR and the vast opportunities available to businesses embracing AR were recently highlighted in a Harvard Business Review article, co-authored by Harvard Business School Professor, Michael Porter, and PTC CEO, Jim Heppelmann.

“We saw the significant benefit augmented reality could have for enterprises years ago, and the market is beginning to catch up and embrace the potential as well. Our recent momentum and continued focus not only strengthens our leadership in this space, but also enables PTC to help companies unlock the potential of augmented reality across all kinds of environments and use cases more effectively,” said Jim Heppelmann, president and CEO, PTC. “Our dedicated augmented reality business unit enables us to sharpen our focus on the application of AR in industrial settings and create additional opportunities for businesses.”

Award-winning Technology and Top Talent Meet Under Vuforia Brand

The dedicated PTC augmented reality business unit combines top talent and award-winning technologies, including the Vuforia^® platform, the industry’s most advanced and widely adopted AR technology platform. PTC has leveraged Vuforia to expand the capabilities of its market-leading ThingWorx^® Industrial Innovation Platform, enabling customers to visualize, instruct, guide, and improve interactions with physical things.

PTC’s augmented reality portfolio includes:

• Vuforia Engine –  enables users to develop powerful cross-platform AR applications
• Vuforia Studio (formerly ThingWorx Studio) – enables users to reuse 3D CAD content, incorporate step-by-step instructions and IoT data, and scale AR authoring and publishing in industrial enterprises
• Vuforia View (formerly ThingWorx View) – with this universal browser, users can consume Vuforia Studio-created content on phones and tablets running iOS, Android, and Windows, as well as digital eyewear like Microsoft HoloLens
• Vuforia Chalk™ – enables technicians to get remote assistance from experts
• Creo^® Design Share Solution – enables users to easily share Creo^® 3D CAD content in augmented reality with partners and other stakeholders

Source: full press release.

It’s a sad fact that plastic debris in our oceans is killing marine wildlife at a staggering rate. With plastics pollution spiralling out of control, shops are under pressure to cut down on their use of unnecessary packaging. Some of the greatest culprits in generating harmful waste that often ends up in the oceans or landfill are cosmetics companies – just think about how many bottles and pots of shampoo, conditioner, moisturiser, shower gels and other grooming products the average person keeps in their bathroom and it’s easy to see how it very quickly adds up.

The increased awareness of the problem among consumers means that this is a commercial opportunity for those offering environmentally friendly alternatives, however. And this is what UK handmade cosmetics company Lush has been doing for many years, making many products such as shampoo in solid form (which according to the company saves nearly 6 million plastic bottles a year) where they require neither packaging nor preservatives and are offered in what the company refers to as “naked” form (where at most buyers have the option to slip it into a small paper bag or have it wrapped in some greaseproof paper).

Currently over 40% of Lush’s product range is completely free of packaging, but in early June the company went a step further and opened a shop in Milan, Italy which is 100% packaging free, relying on technology to keep customers informed instead. The trial is currently running exclusively on Fairphone, and customers visiting the Milan shop have access to four devices, yet they hope to roll it out to global customers in the near future.

Powered by Tensorflow from Google, the Lush Lens app uses AI and product recognition to eliminate the need for packaging. In its current beta version, the information comes up as a simple text pop-up, but there are plans to explore more possibilities around immersive technologies such as Augmented Reality to provide more information about the sourcing of ingredients, the process of making products, provenance about who actually made each batch, and suggestions on how to use it. For those familiar with the company’s rather eclectic and unusual offerings, those features would all make a lot of sense.

With this prototype mobile app we’ve put new technologies such as AI to a good use in our mission to eliminate more packaging and further educate our customers on our unique cosmetics,” Says Adam Goswell, Technology R&D at Lush. “We believe in the ethical use of data, so all the information Lush has is secure and used in a transparent way. We’re increasingly part of the tech community, providing open source solutions where before only monopolies existed. This is in line with a company ethos that aims to give more than it takes, act transparently, push innovation and raise industry standards.

Lush is well-known for its activism, and have – if you’ll excuse the pun – recently got into hot water for one of their more controversial campaigns which was meant to raise awareness against abusive surveillance practices carried out by certain police force branches in the UK. Having spoken to its founders (Lush is very much a family business at its core in spite of its multimillion-pound global turnover) it is clear that politics and product are very much intertwined in their strategy.

In the past few years Jack Constantine – the son of Lush’s Founders Mark and Mo Constantine – has focused on developing in-house technology under the Lush Digital umbrella, including building an in-house R&D team called Tech Warriors. When we last spoke they were enthusiastic about the possibility of using image recognition and AR functionality to allow people to find out about the products in a shop simply by pointing their smartphone camera at them.

The company sees technology as playing a vital role in achieving their commercial-political goals, which translates not only in advocating for digital rights, embracing open-source technologies and adopting ethical sourcing of hardware, but also in their investment in new technologies such as 3D printing, artificial intelligence, image recognition, and immersive tech such as augmented reality.

Technology doesn’t have to be unethical. We believe tech can be built for the greater good and impact positive social change,” Constantine agrees. “Just like its approach to sourcing ingredients and testing products, Lush believes in continuously challenging norms and driving best practice when it comes to business ethics. One of those key principles is the ethical use of data and technology.

On their blog they have recently explored a common enterprise use for AR that many of our enterprise members have been using for years.  Engineers often stumble across new problems and need the advice of specialists for further guidance, but can you imagine how difficult it can be to explain an issue and receive advice on how to resolve it over the phone?

It often means having to send the specialist out, wasting time and money. But what if engineers could share their challenges with others remotely? Augmented reality could hold the key.

BT’s researchers at Adastral Park have been exploring how engineers can live stream their work to their peers using Microsoft HoloLens. The remote engineer would not only be able to see what is going on but could also annotate what they see for the on-site engineer to view through the smart glass device.

The technique could prove particularly useful in training new engineers.  Full article here.

Conventional wisdom holds that technology makes medical care less human, less personal. But watching patient Wanda Touard with virtual reality goggles on her head and controllers in her hands, exploring the inside of her brain for the first time, gives one the distinct impression that nothing could be more personal than this high-tech tool. “Unreal! Amazing!” exclaimed Touard as she virtually traveled along the left side of the bulging aneurysm that required her to have brain surgery in November.

“This is absolutely weird, but absolutely wonderful. My aneurysm looks like a large sweet potato in the middle of my blood vessels. “If this technology had been around six months ago, would I have wanted to see this before my surgery? Absolutely!”

If doctors had heard 10 years ago that one day the very same techies who create video games would be changing the face of surgery, it’s doubtful they would have taken the prediction seriously. But these days, a traditional MRI or CT scan just doesn’t tell the whole story.

Brain aneurysms, which look like bulging balloons along the artery, are due to weak arterial walls and can be exceedingly dangerous. If they rupture, an internal brain bleed can be fatal. That’s why it’s imperative that a neurosurgeon who treats aneurysms has the most optimal view of it before deciding on a course of action. An MRI of patient Wanda Touard’s blood vessels and aneurysm are superimposed on a photo of Buy Now her face at Ochsner Medical Center, a method called ‘augmented reality.’ Touard was treated successfully for the aneurysm using high-tech imaging. Advocate photo by Shawn Fink “This is beyond 3-D printing of the blood vessels and the aneurysm, because with virtual reality, this technology allows me to go inside the blood vessels,” said Dr. Edison Valle, a cerebrovascular neurosurgeon at Ochsner Medical Center. “I can navigate through the patient’s brain just as I would in surgery,” he said.

Doing a “walk-through” ahead of time makes for safer surgery. Seeing the angles of the blood vessels and the depth of the aneurysm helps doctors plan the least invasive and most effective operation, Valle said.

Putting this technology in place at Ochsner was the idea of a neurologist who has a decided affinity for video game technology and the wherewithal to connect tech geeks and medical professionals in an unusual way. “When I moved to New Orleans, I found out that this place is one of the premiere video game development arenas,” said Dr. Korak Sarkar, an Ochsner neurologist. “Twitch is a startup by a founder in Kenner, which televises video games. Amazon just bought this company for slightly under a billion dollars.

“One of the biggest indigenous companies here is called TurboSquid, which is basically an online marketplace for 3-D assets, and one of their co-founders is also on the board of Ochsner,” he said. “Since I go to video game meetups, I acquired some contacts there for people in New Orleans who could do what I wanted them to do and make virtual reality a natural part of the clinical setting.”

Enter Mike Harris, the lead immersive technology developer in New Orleans at CrossComm, a North Carolina based consultancy which creates mobile applications.

It has become his job to turn Sarkar’s dream into a virtual and augmented reality. “If you can imagine an MRI as a stack of individual slices, you can take that stack and via software pull out all of the salient parts,” said Harris. “You now have a collection of data points that can be used to create a 3-D image, but instead of printing them, we’re bringing them into a virtual environment and allowing the physician to manipulate them any way he chooses.” That can mean virtual reality, with goggles, allowing the wearer to “walk” through his or her own anatomy.

Or it can mean augmented reality, in which virtual information is overlaid onto the real world. (Think Pokémon GO, the game everyone played on his phone in which the Pokémon character popped up as part of the real world environment.) “For medical purposes, we can now overlay digital renderings over the patient’s real anatomy,” Harris said. “We now even have augmented reality glasses which project images into your own field of view.”

It’s all a tremendous help in analyzing the individual cases of patients, the doctor said. Dr. James Kalyvas, an Ochsner neurosurgeon specializing in spines, foresees such augmented reality aiding spinal surgery in the near future. By rendering the patient’s spine in 3-D and then superimposing that image on the patient in the correct position on the operating table, surgeons can precisely spot the target area during the operation, Kalyvas said.

Ochsner Hospital is the only facility in the Gulf South using this particular technology, but it’s hoped it will soon become the standard of care. “Has all of this newfangled technology been useful?” asked Valle. “I’m now able to adjust what I thought I could do in a surgery to what I can really do, now that I can see absolutely. It has blown my mind.

This is because the company is extending the field of activities used by the pick-by-vision solution in the past at its logistics centre in Kirchzarten. Up to now, the warehouse specialists have used ten pairs of the glasses for picking outgoing goods – now the glasses will be deployed for incoming goods too. The electronics wholesaler is hoping to achieve a significant reduction in the error rate when putting items into storage by using the Picavi products in this area.

Currently, ten pairs of smart glasses are in use from 7 a.m. until 8 p.m. and enable the warehouse specialists to pick items like resistors, semiconductor diodes or capacitors for dispatch from the warehouse stocks that comprise 2.5 billion components. The recipients of the packages sent from the ECOMAL central warehouse, which is located in the southern Black Forest area, include companies in the automotive sector, the semiconductor industry or those assembling circuit boards.

“We’ve been able to achieve time savings of at least 20 percent by using the Picavi smart glasses for our multi-order picking system covering different products. As part of the ongoing digitalisation of our warehouse management procedures, we’ve purchased even more models to switch our incoming goods to the pick-by-vision technology too,” says Peter Kraus, Logistics manager at ECOMAL. “We’ve made one or two mistakes in the past and they’ve caused a fairly long search process. This will now be a thing of the past when we put items into storage with the Picavi smart glasses; they indicate where the correct warehouse position is on the display and the relevant information is then stored in the warehouse management system.”

“ECOMAL is a clear example of how we can continue developing a partnership, which started in one area of a warehouse, into a different one. The strictly visual process controls of the pick-by-vision solution can be used for many activities in internal logistics in order to increase the precision of the work and make time savings when picking items,” says Johanna Bellenberg, director of Marketing and Communications at Picavi, underlining the benefits.

Amid the whirring cacophony inside a coal-fired power plant, Eric Zakszewski peers into the side of a coal feeder to see that fuel is flowing properly on its way to a pulverizer. On this day, a flashlight isn’t his only tool. A small monocle hangs from his white hard hat in front of his right eye. On it, he can view a list of tasks and safety checks he is to perform without having to turn to a screen or clipboard.

“It’s like a powerpoint that is hovering in mid-air,” said Zakszewski, a 29-year-old power plant operator at We Energies’ Elm Road Generating Station near Milwaukee.

Zakszewski is among a group of utility workers who recently participated in an industry-backed study looking at the potential benefits and drawbacks of equipping employees with augmented reality (AR) devices, which layer virtual information into a user’s field of vision. A team from the Electric Power Research Institute (EPRI) visited the We Energies coal plant last week to observe workers while they tried out various products.

“The potential uses of AR in the electric utility sphere are already in play,” Eric Bauman, a senior technical leader with EPRI, said. “It is being demonstrated for inspection tasks, performing work, training, and probably more. There is substantial interest amongst utilities in this technology.”

The technology is maturing and coming down in price as the utility industry faces a looming workforce challenge that will require it to hire and train thousands of workers in the coming years as aging workers retire. Still, more vetting is needed to make sure the devices actually help workers and also don’t create dangerous distractions.

AR: “It could be really useful”

Zakszewski performed a series of jobs inside and outside the We Energies plant last week while wearing two types of AR devices — RealWear, the monocle lens, and Microsoft’s HoloLens, a set of glasses.

“For the most part, my job is looking and observing,” Zakszewski said. “So, AR does obstruct it a little, but if you could have it off to the side it could be really useful.”

Besides scrolling through checklists, the devices will become more useful as more meters, poles, and other grid equipment is wired into the internet of things. With AR glasses, all that data streaming from these smart devices can be visualized, which is “absolutely part of the allure of AR,” Bauman said.

Brian Dixon, chief operating officer of Capital Innovators in St. Louis and head of the Ameren Accelerator, an energy technology startup accelerator, agreed. “Imagine somebody working for a utility in the field is able to look at the transmission wires or different types of large assets and get a perfect understanding through the data that’s being projected into their field of vision,” Dixon said.

Ameren, one of Illinois’ largest utilities, is among the companies exploring how AR gadgets can bring efficiency and safety to its operations.

“Ameren sees significant long-term potential in applying augmented reality technology to maintenance operations, as well as other aspects of our business,” said Steve Kidwell, Ameren’s vice president of corporate planning.

Over time, Kidwell predicts the devices will reduce costs and improve service while helping to train and keep workers safe in the field.

Products cost less, perform better

A 2016 EPRI report that reviewed existing research literature concluded that affordability is no longer an obstacle for utilities, as costs for the devices have come down. “More and more players are entering the marketplace. It is a vibrant area of technological development and experimentation,” Bauman said.

Vuzix, a developer based in Rochester, New York, is creator of the Blade, a $1,000 pair of AR glasses that work with Amazon’s voice-controlled Alexa platform. Designed to look like sunglasses, the Blade has received high praise from the technology press — The Verge called it the “next-gen Google Glass we’ve all been waiting for.”

But the company also produces the M-300, a sturdier device designed for industrial use that runs about $1,500. Instead of looking through glasses or a large pair of goggles, a user looks through a small monocle that mounts onto a hard hat. The company’s chief executive envisions a role for the product in the utility industry.

“You can literally just look at the meter, and have the meter reading captured on the device,” Paul Boris, Vuzix CEO, said. “You don’t get errors in transcription, you don’t have to tap the reader, you don’t even have to get close to it — you can literally be on the other side of the fence.”

Meanwhile, other utility hardware is getting smarter, expanding the potential uses of AR glasses to interact with equipment.

Ontario-based Hyperion Sensors, one of the first companies in Ameren’s startup incubator, developed fiber-optic sensors the size of human hair that can be strung into transformers and other high voltage equipment, making the “dumb” equipment smart and providing instant information about temperature and strain.

“Over the next 20 or 30 years, the idea is that this smart technology will be part and parcel of everything that gets connected to the grid,” Anselm Viswasam, CEO of Hyperion Sensors, said. “If you are going to use a solar panel and it’s going to supply power onto the grid. Then it will be necessary that it will have the smarts embedded into it.”

A tool for looming workforce challenge?

The utility industry faces some significant workforce challenges ahead as the baby boomers retire, which was highlighted by a 2017 Department of Energy utility workforce assessment. Few utility workers have the training or qualifications to move up the ranks as older workers retire, it concluded.

“Workforce retirements are a pressing challenge. Industry hiring managers often report that lack of candidate training, experience, or technical skills are major reasons why replacement personnel can be challenging to find—especially in electric power generation,” the Energy Department report said.

We Energies plant central scheduler Randy Sheck sees potential for using augmented reality to train and assist new workers. Utility workers routinely work 12-hour shifts, walk 5 or 6 miles and examine 300 pieces of equipment.

“You always have to be looking for new technology,” he said. “I think augmented reality may still be 5 or 10 years out, but it has a lot of potential. I see the value it can add for us in terms of daily maintenance.”

Nakazawa strives to illuminate how to drive leading experiences where the human, digital, and physical intersect, and how we can re-invent ways people come together to create, consume, and celebrate the digital experiences that constitute life as we know it, feel it, see it, and immerse in it.

In this interview Nakazawa talks about Atheer, their products, history and news, and goes on to discuss their working partnership with RealWear.

The full interview appears here.  See also the AREA profiles of Atheer and RealWear.

No. 1 – The world’s best-known brands will use enterprise AR to create and deliver their products and service their customers. Delivered via wearables like smart glasses, AR promises new levels of efficiency, service and quality that appears not only on the plant floor, but also throughout the entire supply chain. We are seeing large manufacturers moving beyond the pilot phase and adopting AR as part of a broader digital transformation strategy. In fact, PWC expects that 1 in 3 manufacturers will adopt AR/VR this year.

No. 2 – Service and logistics will driver broader AR adoption. Largely due to the intrinsic repeatability of their AR applications, field service and logistics are helping to move the market forward. This means that manufacturers should explore AR use cases involving their warehousing, material handling and equipment maintenance operations, where even a 10 percent efficiency gain can impact the bottom line or the ability to move production at a higher rate. For instance, workers can use AR on smart glasses to receive remote expert guidance during unplanned downtimes, or to attain step-by-step instructions during the picking process.

No. 3 – AR toolkits will simplify content creation. With the maturation of AR software development toolsets, there is a lower entry barrier for the knowledge required to build AR applications. Current toolkits offer drag-and-drop interfaces and templates, requiring little to no programming prowess. Between AR experiences being driven from Manufacturing Execution Systems and low/no-code authoring tools providing ways for designers and technicians to participate in the adoption and fine tuning of augmented reality, the time and complexity of adoption drops dramatically. With typically “non-technical” people creating new AR experiences, AR adoption will accelerate in the manufacturing realm and beyond.

No. 4 – Voice will be the primary AR interaction paradigm. The paradigm of Voice In, Image Out will be common in the age of AR. Gesture and touch AR interfaces aren’t exactly ideal for applications involving hands-on work, especially on the manufacturing plant floor. To keep workers free and nimble to perform their jobs, voice is emerging as the preferred method of AR interaction. We already see voice-powered smart assistants, like Amazon Alexa, gaining traction in consumer realm, so using voice in the enterprise is a natural trajectory, as many workers are already familiar with carrying out tasks using the technology.

No. 5 – Consumer AR investments will drive increased device options and experiences. There are many more exciting AR developments to come as adoption drives more investment into the tools and technology behind the AR industry. Behemoths like Google, Microsoft, Amazon, Apple and Facebook are unveiling new AR toolkits and decimation tools that will lead to an explosion of AR, VR and MR content. For manufacturers, the availability and accessibility of this content will raise the bar (and expectation) for how we engage with workers through rich, real-time information that enables them to complete their tasks faster and with greater quality. The sky will be the limit.

Looking back at these five trends, it is important to note the underlying theme: AR in manufacturing is all about the future of work. Despite today’s emphasis on automation and robotics, people will continue to play a pivotal role in hands-on work. AR will bridge the gap between man and machine, equipping workforces with right information, at the right time, and in an actionable manner. Trends such as new toolkits, new use cases, new interaction paradigms and new content will continue to accelerate AR adoption – empowering workers and laying the foundation for digital transformation across entire organizations.