AREA member NGRAIN started in the enterprise training market and today is an AR solutions provider for a range of companies and industries.

We recently interviewed Barry Po, NGRAIN’s Senior Director of Product and Business Development, to discover the latest developments about NGRAIN’s offerings for industrial Augmented Reality.

What is your company’s mission and focus in the market?

NGRAIN has been working with customers to prepare and publish training programs and other types of information in rich and engaging ways for over 15 years. We develop solutions using both Augmented Reality and VR to meet the needs of our customers in aerospace and defense, energy and utilities, oil and gas and manufacturing and healthcare.

In each of these industries there are specialists who work with physical objects—whether to deploy, operate or to maintain and service these machines—and who need the right information in the right place at the right time. That’s what Augmented Reality brings: the ability to access information that would otherwise not be readily available or easy to understand, and equipping these people with knowledge they need to make better decisions. As a result, training time is shortened and they can perform tasks quickly and correctly every time it’s required.

A field technician’s work is often more complex than outsiders understand. When preparing and executing some tasks, there is a staggering number of details. Many human errors happen when working with heavy assets, such as maintaining or operating heavy equipment like a vehicle or a complex assembly. The value of Augmented Reality in those situations is to reduce errors, as well as cut down on missteps and omissions of technicians in the field. The technology makes it more practical for someone to do a complex job and ensures that they don’t forget anything along the way.

Another major benefit is that a comparatively less experienced person can use the information without having to spend time in the classroom before becoming productive.

What products and technologies does NGRAIN offer?

We offer a full suite of solutions so that the customer can reach the results they seek quickly. Our AR software development kit allows customers to build custom applications with AR. NGRAIN Producer Pro is what people who want to author their own AR applications with a GUI use. It doesn’t require programming experience. It allows authors to create or import their 3D content and to link it to metadata, as well as display it on Windows, iOS and Android mobile devices. Lastly, NGRAIN also provides customized Augmented Reality solutions tailored to customers’ specific needs.

Are there some use cases that, in your experience, are particularly well-suited for AR?

One major use case that NGRAIN addresses is maintenance training. Our AR-enabled solutions help someone in the field learn on the job rather than just in the classroom. It helps them figure out what they need to do, as well as what’s needed for their work and to get feedback. Having it all on a mobile device such as smart glasses makes it easily accessible.

Another major use case is visual inspection and damage assessment. Our solutions for battle damage assessment and repair are deployed in the field by Lockheed Martin, which has been an NGRAIN customer for eight years. US Air Force technicians use our technology to assess and organize repairs for F-22 and F-35 aircraft. This maximizes the amount of time the aircraft spend in flight and reduces maintenance costs and time spent in the hangar.

Which measurements or metrics for assessing AR’s impact do you prefer?

From an AR perspective, our customers are in the process of defining business cases and metrics, so measurements such as ROI have yet to be defined in a standardized way.

If we take a broader perspective that includes Mixed and Virtual Reality however, we can make a few generalizations about KPIs. Based on NGRAIN’s experience deploying 3D applications for maintenance training, we find the technology can double knowledge retention, which, in turn, brings a variety of benefits. For example, technicians become less prone to missing steps or mixing up the sequence. This increased efficiency also enables them to focus more on the job as a whole and ensure it’s well executed.

A third interesting metric is a statistic measuring how often a job is completed correctly the first time it’s performed. When we deploy the technology, we find customers are able to execute the job correctly nine times out of ten. In the oil and gas industry, for example, correct first-time job execution only occurs 30-40% of the time, so the technology’s impact can be significant.

What is your approach to introducing customers to AR?

We look at the customer’s problems first, focusing on their business environment and organization. It’s important to understand a customer’s pain points in achieving their goals, and one way we do this is by spending time at their sites and observing their operations firsthand.

Recently we spent time with an oil and gas customer’s technicians in the field that was maintaining drilling equipment. We learned that much of the knowledge needed to correctly do jobs isn’t actually documented, but nevertheless is subsequently required by less-experienced people. Our aim, with our 3D guidance solutions, is to provide this kind of tribal knowledge as a virtual mentor might.

What are the typical customer organization’s approaches with respect to new technology introduction?

Everyone agrees that technology is a valuable part of any organization, but we often find differences of opinion in the degree of intensity that new technologies should be introduced. For example, many people who would benefit from AR really don’t care about the technology itself but are looking for the efficiency gains it provides.

In our view, introducing new technologies is less about imposing an approach on the customer or the end user. We make them a part of the process of discovering what works best for them. This ensures that everyone’s perspective is taken into account in the process, rather than the process being solely about the vision of a person or small group of people at the top.

A successful deployment of AR technology takes effort and is unique to each customer and group. Discovering the right approach for a particular customer is greatly helped by working with stakeholders at all levels.

Wearables are leading the next industrial revolution as enterprises leverage their data and systems and employees work with the physical world in new ways. From sensors in clothing to smart watches and glasses, wearables will connect the modern workforce with data and with one another. Wearables are also becoming a component in the emerging constellation of tools for accessing the Industrial Internet of Things.

One class of wearable devices—namely smart glasses—are also essential to delivering Augmented Reality experiences that promise efficient, hands-free work, without having to switch attention to a paper manual or hold a mobile device. Smart glasses are evolving rapidly so business cases and implementation practices must advance as well.

New forums are offering answers that help executives and representatives of companies put wearables to use and to grasp their full potential. AREA member BrainXchange will organize and present the Enterprise Wearable Technology Summit East, a conference on wearables in enterprise and industrial settings, in Atlanta, Georgia, on June 16 and 17.

Putting Enterprise Wearable Practitioners First

Many events provide vendors the opportunity to present their wares to customers. There’s a lot of value in this, but it’s not the complete picture.

Enterprise Wearable Technology Summit East focuses on real world case studies. By sharing the experiences of pilot projects and production implementations, participants have the opportunity to compare experiences about this new wave of technology and formulate best practices.

This isn’t to say that vendors of wearable products and technologies are not welcome. On the contrary, they need to understand customer frustrations as well as achievements. Indeed, the team at BrainXchange relies, in part, on the vendors of wearable technologies to open the doors to customers who may not feel comfortable in the limelight.

Diversity Fuels Investments

With only pockets of successful wearable deployments in any one industry, any industry-specific solutions can’t be justified. The many industry-specific conditions and opportunities will influence the pace of adoption of wearables within one vertical. However, a healthy multi-industry dialog can create a larger market pull that could be sufficiently big to fuel new investments.

Attendees from diverse industries such as oil and gas, healthcare, utilities, shipping and others will attend EWTS East to discuss and gain insights on the data and lessons learned on the ground and in the trenches.

From the presentations and panels on the stage and the private sidebar conversations that begin in the context of networking functions, patterns will emerge. AREA members will be sharing their insights and opinions.

AREA-Moderated Group Discussion

AREA Executive Director Christine Perey will moderate the second-day group discussion on “Cultural and Organizational Considerations” of wearables, which will discuss business challenges of wearables faced by enterprises, including:

• Privacy and ethics
• Workplace culture and behavioral changes
• Costs and ROI
• Prospects of standardization and regulation
• Safety concerns of wearables by industry

The panelists and audience will gain new perspectives about the rise of wearable computing and how it will promote new human-world interactions and connect us more intimately with the surrounding environment.

Are you going to attend the EWTS East event in mid-June? Join the AREA and promote your participation in this special event in Atlanta.

If you only have a few days to get up to speed about the use of Augmented Reality to improve workplace productivity or safety, then you’ve come to the right website. You’ll quickly pick up the concepts and a working vocabulary of AR by browsing our site and watching our webinars. You’ll also learn about our members, leaders dedicated to providing AR-enabling technologies and solutions who offer a wealth of knowledge and experience.

But, for most people immersing themselves in this exciting new discipline, surfing the web isn’t sufficient. Let’s face it: enterprise AR experiences always involve a physical world component.

Putting hands on the technologies while they’re in use, to see different options and meet people in person, are critical to tapping the potential that enterprise AR offers.

AWE Brings the AR Industry Together

The upcoming Augmented World Expo (AWE) on June 1 and 2, 2016, in Santa Clara, California, is an important event for AREA members. Over four thousand AR practitioners and enthusiasts will experience the latest technologies when gathering important data for decisions on behalf of their companies and projects.

Organized annually since 2010 by industry mover-and-shaker Ori Inbar and his team, the event has both a conference and a trade show. These bring together customers, vendors, researchers, investors and many others who are important to the continued expansion of this industry in a variety of formats. 

Whether you’re looking for something specific or just exploring, AWE provides an opportunity to get to know the experts, such as AREA members, and to try out the latest Augmented Reality technologies and products first hand.

AREA Members at AWE

The AREA and its members will be leading and speaking during the AWE 2016 enterprise AR track of sessions, as well as demonstrating solutions on the exhibition floor.  

In order to provide the greatest impact to our diverse audiences, we’ve divided the enterprise AR sessions into vendor-neutral insights and recommendations from a range of technology providers on June 1, followed by customer case studies and testimonials on June 2.

Chaired by Paul Davies, Technical Fellow at Boeing, an AREA founding sponsor member, our June 1 speakers will provide a lot of practical advice based on their experience across many industries. Since it is frequently the first major barrier to success, the day will begin with speakers sharing recommendations about how to select and prioritize enterprise AR use cases.

Then AREA members will offer their suggestions for how to prepare and deliver digital content for enterprise AR experiences. In this session, I will provide the results of a research project on different AR authoring platforms. David Marimon of Catchoom will describe the results of recent studies with 3D sensing platforms for real world object recognition and Alex Hill, CTO of CN2 Technologies will offer guidance on how to optimize 3D assets for use in AR experiences.

The rest of the day promises great talks on enterprise AR wearable technology strategies and the use of AR as a human interface to Industrial Internet of Things.

On June 2, Bob Meads, CEO of iQagent, another AREA founding sponsor member, will chair three hours of sessions during which customers will share their experiences working in pilot and proof-of-concept projects. These sessions will feature case studies and testimonial presentations. The afternoon will offer round tables and panel discussions with customers, and we’ll hear the results of recent projects and lessons learned throughout the day.

AREA members will also be exhibiting in record numbers and many will have their booths in the AR for Enterprise Pavilion.

Will you be there? Stop by AREA member booths to introduce yourself to us and let us know how we can help you to get the greatest value from attending AWE 2016 and your enterprise AR investments.

In a previous post we described how, by developing a new framework that leverages Augmented Reality, IoT, social networking and advances in hardware, the members of the European SatisFactory consortium seek to increase productivity in manufacturing.

After a period of design and development, SatisFactory solutions and technologies will be validated at three pilot manufacturing facilities.

Each of the three pilot sites corresponds to a different industry:

• Chemical Processes: The Chemical Process Engineering Research Institute (CPERI) is a non-profit research and technological development organization based in Thessaloniki, Greece.
• Industrial Automation: Comau S.p.A is a global supplier of industrial automation systems and services based in Turin, Italy.
• Energy: Systems Sunlight S.A. headquartered in Athens, Greece, is a manufacturer of energy storage and power systems for industrial and consumer applications.

Installation and validation of the SatisFactory framework at each of the sites is an iterative process, with Augmented Reality devices (with technology partner GlassUp) and other technologies to be implemented in the coming months.

While this article focuses on plans for Augmented Reality at each of the three pilot sites, there are many more technological aspects bringing together innovations for streamlining efficiency in the factory. A publicly available project report sheds light on the use cases described below.

Continuous Production in Chemical Processing

CPERI is an institute that performs research and provides services to industries related to chemical engineering, energy and materials. CPERI is an ideal site for testing and improving continuous processes. In contrast to batch manufacturing of goods, continuous processes for chemical, pharmaceutical, food processing and other types of plants impose different challenges. As the facilities use equipment that must run continuously, any downtime can be costly. When shutdowns occur, incomplete products must often be disposed of, and the corresponding infrastructure (e.g., pipes, vessels, etc.) thoroughly cleansed of remaining materials.

Startup and shutdown procedures must be validated and documented to prevent all unwanted impacts. Typically such procedures require several hours to complete. CPERI is pioneering Augmented Reality in a use case for plant startup procedures, in which an operator using an AR device completes a task normally requiring many sequential steps done in several hours for starting up a plant.

The AR-enabled system incorporates a human-machine interface to display real time feedback to the operator from the Supervisory Control and Data Acquisition (SCADA) automation system and other process control systems. Without AR, such tasks must be performed by an experienced operator referencing a manual, while the new AR solution proposes a workflow with reduced attention switching and can be performed by someone with less prior experience.

Additional use cases for AR are being designed and discussed with other SatisFactory pilot sites. CPERI, along with its partners, are also authoring a set of standard operating procedures for enabling and using Augmented Reality to improve productivity and compliance in continuous manufacturing and other chemical industrial processes.

Discrete Manufacturing in Heavy Industry

Part of the Fiat Group, Comau provides industrial automation systems for manufacturing in the automotive, aerospace, steel and petrochemical industries. The company specializes in:

• Body welding equipment for a variety of vehicle types
• Manufacturing systems for engine powertrain components
• Robotic systems for a range of manufacturing use cases

Comau is developing an Augmented Reality-enhanced system to help a user assemble a robot wrist, a process that normally requires four hours and over 290 individual steps to complete. The proposed AR solution, to be provided on a device that’s either fixed or wearable (e.g., smart glasses), uses animations that appear over the real world to guide the technician through steps to complete the task in less time and with fewer errors than existing methods.

As with CPERI, the AR solution will eventually be integrated into the site’s IT infrastructure, including warehouse management, enterprise resource planning and manufacturing execution systems to provide real time data and support.

Remote operator assistance and technician training are other use cases being evaluated. Operators will be able to request live help and collaboration from remote experts for tasks, as well as choose training scenarios from online repositories for on-the-spot examples and guidance. Comau is also evaluating how AR can be used to record processes performed for verification and future training purposes.

Augmented Reality in Power System Manufacturing

Systems Sunlight’s integrated energy products span a range of industries from vehicles and consumer electronics to utilities and defense. The company operates a manufacturing facility where assembly lines produce batteries, transforming raw materials into ready-to-use products. Battery production requires continuous monitoring of variables such as cell temperature, which is measured with a thermal camera.

Augmented Reality is being evaluated for producing “motive batteries” for powering machinery: in a two-hour assembly process of six major steps, an operator places battery cells in a metal box whose sequence depends on battery type. The cells are then connected by means of a battery string and later checked whether they need additional electrolyte filling. They are then sealed with regular or water-filling plugs. In the next two stages, the terminal plugs and labels are installed and the batteries are checked for quality. Finally, the batteries are transported to a warehouse for dispatch to customers.

Systems Sunlight will implement Augmented Reality guidance on a fixed or wearable device so the technician can work hands free. Besides the expected benefits that animated steps overlaid on the field of view can provide, the company anticipates the technology will increase the overall motivation of technicians and operators. They plan to measure this increase with surveys after the conclusion of pilot testing.

If the pilot is successful, the company will explore use of Augmented Reality on further production lines and create a training system combining AR and gamification, along with quizzes to reinforce knowledge.

A Template for Manufacturing Efficiency

The lessons learned in these three pilot project sites will allow the SatisFactory solutions to be fine-tuned and demonstrate their value for the manufacturing sector.

In 2017, they will be made available to the European manufacturing industry to improve efficiency through novel interaction and collaboration technologies. The solutions also aim to improve the quality of life and overall working experience of factory operators, and mark a major step forward in European manufacturing competitiveness.

Industry watcher and analyst firm Gartner has been studying emerging technologies for over 20 years. The company has become widely recognized for publishing its annual Hype Cycle, the chart that captures Gartner analysts’ assessments of the maturity of emerging information and communication technologies.

Interpreting the Hype Cycle

As stated on Gartner’s website, the chart is designed for the firm’s clients : “Clients use Hype Cycles to get educated about the promise of an emerging technology within the context of their industry and individual appetite for risk.”

The sidebar on the same page goes on to suggest that the Hype Cycle:

• Separates hype from the real drivers of a technology’s commercial promise
• Reduces the risk of your technology investment decisions
• Compares your understanding of a technology’s business value with the objectivity of experienced IT analysts

In my slides introducing the March AREA webinar on the topic of forecasting the growth of enterprise Augmented Reality, I provided 15 Hype Cycle figures of the years between 2000 and 2015 showing where Gartner placed Augmented Reality. These figures were compiled by Dr. Robin (Rab) Scott of the AMRC, an AREA member, and used in the webinar with Dr. Scott’s permission.

The figures show how this influential firm has followed Augmented Reality for over a decade. I pointed out in my remarks that readers should not interpret the position of any technology on the Gartner curve as highly definitive.

Looking at Gartner’s positioning of Augmented Reality over the years, and anticipating the 2016 Hype Cycle to be published, I am recommending in this post that Gartner consider treating Augmented Reality and its associated technologies as separate nodes on the cycle. By giving more attention to AR’s enabling technologies Gartner will help its clients better achieve their goals and better serve our industry.

Augmented Reality Isn’t One Technology

My primary concern about Augmented Reality appearing as a dot on the Gartner 2015 Hype Cycle is that it suggests that Augmented Reality is one technology. I don’t think this was ever the case in the past and it certainly isn’t today.

In its press release about last year’s Hype Cycle, the company stated that more than 2,000 technologies were studied. It would be helpful if the firm pointed out which of the hundreds of AR-enabling technologies it considered in positioning the “whole AR” on its cycle.

In my opinion Gartner needs to begin explaining how technologies are treated differently. For instance, some technologies on the cycle are “general” (representing many enablers at different stages of evolution), and others are not. In 2015, for example, brain-computer interfaces are in the first phase. Gesture control technologies, another relatively precise technology label, are on the slope of enlightenment. Another example is natural language question answering (very specific technology, in my framework but probably also composed of many enablers), which is positioned on the line between Peak of Inflated Expectations and the Trough of Disillusionment. And, by the way, when will the questions asked be answered correctly all the time?

On the other hand, Augmented Reality is not the only example of the ambiguity and confusion caused when a general category is represented as a dot on the cycle. For example, wearables and Internet of Things are other labels (represented as dots) on the 2015 Hype Cycle that could benefit from being represented by an array of enabling technologies (or are they enablers?).

In my opinion, the company would better serve its clients and readers by tracing the progress of some of the important enablers or components for AR and other technologically-powered systems, such as autonomous driving vehicles. A few components that I have recently studied for a technology maturity assessment, and that I believe should be added to the Hype Cycle, include:

• Depth-sensing technologies
• Computer vision-based 3D target object recognition and tracking
• Optics for use in wearable displays
• Gaze detection and tracking technologies

Enlightenment Is a Process

Enlightenment about the benefits of a technology does not happen by simply turning on a light. The processes by which technologies move from barely understood to mainstream use differ widely.

Twenty-five years ago I began reading and writing about the future with multimedia information. Multimedia was not on the Gartner curve in 2000 because it had already reached something approaching maturity; now it’s an archaic term. I have been an outspoken proponent for the adoption of mobile technologies for over 12 years. Mobile technology was not a dot on the Gartner curve in 2004 but its enablers such as MMS and 802.11 g certainly were.

Would it not be better to leave Augmented Reality off of the 2016 and future Hype Cycle figures and, rather, to point the spotlight on the state of dozens of key enablers?

Do you feel the Gartner Hype Cycle correctly portrays the state of Augmented Reality? What would you like to see added or removed from the Gartner Hype Cycle in 2016?

For decades, telephones have been a mission-critical business tool for reducing the need for close proximity of experts and those they support. Internet- and web-based videoconferencing is known to contribute to lower travel-related costs and delays in delivering training and technical support.

As the quality of connectivity, cameras and other components have improved in recent years, the use of personal or company-issued mobile devices by employees for technical support has flourished. In some industrial and enterprise settings, mobile devices are required for rapid access to work orders and instructions, as well as capturing issues and documenting conditions. In many situations, capturing one or more high-resolution photographs when documenting an issue is faster and more complete than composing a written description. When a question or problem is quickly grasped by an expert who is not physically nearby, the expert can avoid travel but not delay the rapid resolution of the issue by proposing the most appropriate steps.

While photos of issues in the field can be beneficial, they aren’t always sufficient to describe a problem fully. And experts reviewing the photos are unable to clearly communicate their findings in a visual manner.

Researchers in academia and industry have studied these problems for over a decade and developed a variety of prototypes to enable clearer communication. Some commercial solutions were proposed prior to 2015 but they were not successful for a variety of reasons.

Needs and Technologies Align in 2016

Customers’ needs and a maturing array of enabling technologies are creating the conditions for AR-enhanced remote assistance to take off in 2016. In the past 12 months alone, dramatic improvements in AR-enriched annotation capabilities in real time for mobile devices and even wearables are visible.

Customers who have implemented pilots in 2014 and 2015 are beginning to share positive results and rapid return on investment with few significant obstacles. They also have increased their abilities to capture tribal knowledge and to train new operators in best practices without sacrificing productivity.

In response to this trend, investments have also markedly increased. Today there are dozens of suppliers in this segment, with solutions ranging from prototypes to battle-tested commercial packages. Matching options to needs will be more challenging in the future without a clear understanding of trade-offs and benefits.

A Vendor-Neutral Technical Report

The AREA’s new technical report is designed to support the dialog between customers and suppliers, and for suppliers to better understand where they can offer the most value to partners.

The AREA report begins with a snapshot of a typical AR-enhanced remote assistance system architecture and describes four generic use cases that apply to hundreds of specific scenarios across dozens of industries. It offers a simple landscape of companies in this area, separating those currently providing technologies for possible integration into a complete AR-enhanced remote assistance system, to those providing complete solutions for remote assistance. In addition, the report’s appendix summarizes the literature published in engineering journals on the topic of remote assistance with Augmented Reality.

Although some of our members provide solutions for AR-enhanced remote assistance, the study is impartial and neutral. AREA members have exclusive access to the full report. On this date, it will be available publicly for use by anyone seeking further insight and understanding of the opportunities and providers.

A Rapidly Changing Market

The AREA’s technical report aims to be an introductory tool for those who are getting familiar with the segment rather than an exhaustive and detailed buyer’s guide. As the level of investment and attention to this segment continues to rise in coming months, the technologies and suppliers are rapidly evolving. New features are emerging with each release. And, although the report does not examine this aspect, new and different business models are being proposed and tested.

Have you implemented projects that introduce AR-enhanced remote assistance to improve collaboration between experts and field technicians? Are you performing research to compile all up-to-the minute details about precise solutions for remote assistance? What unique requirements do you have?

Fulfilling customer orders at a warehouse, or order picking, can be costly. A well-known study on warehouse management cited the typical costs of order picking as being nearly 20% of all logistics costs and up to 55% of the total cost of warehousing. The use of technology to streamline order picking offers an important opportunity to reduce cost.  

While great strides have been made in automating warehouse processes, customer expectations also continue to rise. For example, Amazon offers same-day delivery in many US metropolitan areas and this is becoming a standard elsewhere. Increasing fulfillment and delivery speeds may result in increased errors that are not caught prior to shipment.

Augmented Reality can significantly increasing order picking efficiency. An AR-enabled device can display task information in the warehouse employee’s field of view. Logistics companies such as DHL, TNT Innight and others have been collaborating with providers of software and hardware systems to test the use of Augmented Reality in their warehouses.

A recent study by Maastricht University conducted in partnership with Realtime Solutions, Evolar and Flos brings to light the impact smart glasses can have on order fulfillment. The research sought to:

• Confirm prior research that smart glasses improve efficiency compared with paper-based approaches
• Study usability, required physical and mental effort and potential empowerment effects of the technology in a real world environment
• Assess the impact of an individual’s technology readiness on previously introduced performance and well-being measures

Design of the Study

Sixty-five business students at the University of Maastricht participated in a three-day study conducted in a controlled environment. Study participants were given instructions to pick individual items from bins containing items and place them into appropriate customer bins:

• One group picked items from 28 bins using item IDs printed on paper and then matched those to IDs on customer bins. The study assessed order picking efficiency by measuring the ability and speed of participants to place the items in the correct customer bins.
• The other group used AR-enabled smart glasses to scan barcodes in item bins and follow the displayed instructions to place them in the customer bins.

The researchers evaluated metrics such as:

• Performance measures of error rates and picking times per bin
• Health and psychological measures such as heart rate variability, cognitive load and psychological empowerment
• Usability measures such as perceived ease of use
• “Technology readiness” on a scale measuring personal characteristics such as optimism for, and insecurity with new technologies

Faster with Smart Glasses

The researchers found that smart glasses using code scanners permitted users to work 45% faster than those using paper-based checklists, while reducing error rates to 1% (smart glasses users made ten times less picking errors than the control group).

The smart glasses group also expended significantly less mental effort to find the items with the same heart rate variability as the group using paper.

Overall the usage of smart glasses empowers users and engenders positive attitudes toward their work and the technology: in comparison with the group following checklists, they felt the successful completion of tasks was more attributable to their own behavior. This corroborates other studies in efficiency gains such as this one, and demonstrates the level of impact Augmented Reality can have in the workplace.

You can read about more Augmented Reality research from Maastricht University and other university partners at this portal.

In a blog post last month, Giuseppe Scavo explored the Industrial Internet of Things (IIoT) and the growing trend of connected devices in factories. Smart devices and sensors can bring down production and maintenance costs while providing data for visualization in Augmented Reality devices.

Connecting AR and IIoT requires applied research. In this article we’ll look at the EU-sponsored SatisFactory project, which is focusing on employee satisfaction in factories by way of technology introduction.

Innovation in Industrial Production

In 2014, the European Union launched Horizon 2020, a seven-year research and innovation program (ending in 2020) dedicated to enhancing European competitiveness. Horizon 2020 is a partnership between public and private entities and receives nearly $90 billion in public funds. As the program’s website describes, Horizon 2020 aims to drive smart, sustainable and inclusive growth and jobs.

Within this push is the Factories of the Future initiative, a roadmap providing a vision and plan for adding new manufacturing technologies to the European production infrastructure. Objectives of Factories of the Future initiative include:

• Increasing manufacturing competitiveness, sustainability, automation
• Promoting energy-efficient processes, attractive workplaces, best practices and entrepreneurship
• Supporting EU industrial policies and goals

To meet these objectives, ten partner companies and institutions from five European countries founded the SatisFactory consortium in 2015. SatisFactory is a three-year project aiming at developing and deploying technologies such as Augmented Reality, wearables and ubiquitous computing (e.g., AR-enabled smart glasses, etc.) and customized social communication and gamification platforms for context-aware control and adaptation of manufacturing processes and facilities.

SatisFactory-developed solutions seek higher productivity and flexibility, job education of workers, incident management, proactive maintenance and above all a balance between workers’ performance and satisfaction. The solutions are currently being validated at three pilot sites (one small- and two large-scale industrial facilities) pending release for use at industrial facilities throughout Europe.

Industry 4.0

SatisFactory’s vision of Industry 4.0 includes a framework with four sets of technologies:

• Smart sensors and data analytics for collecting and processing multi-modal data of all types. The results of this real time data aggregation will include diagnosing and predicting production issues, understanding the evolution of the workplace occupancy model (e.g., balancing numbers of workers per shift) and enhancing context-aware control of production facilities (e.g., semantically enhanced knowledge for intra-factory information concerning production facilities, re-adaptation of production facilities, etc.).
• Decision support systems for production line efficiency and worker safety and well-being. These systems can take many forms, ranging from Augmented Reality for human visualization of data to systems for incident detection and radio frequency localization.
• Tools for collaboration and knowledge sharing, including knowledge bases and social collaboration platforms. Augmented Reality for training by remote instructors will provide flexibility and increase engagement. Collaborative tools also allow employees to exchange information and experiences, and these tools are combined with learning systems.
• Augmented Reality and gamification can increase engagement. SatisFactory will use tools previously developed by consortium partners and, in pilot projects, explore use of smart glasses and human-machine interfaces. Interaction techniques and ubiquitous interfaces are also being explored.

Pilot Sites

SatisFactory solutions are being tested at the pilot sites of three European companies:

• The Chemical Process Engineering Research Institute (CPERI) is a non-profit research and technological development organization based in Thessaloniki, Greece. The company provides a test site for continuous manufacturing processes.
• Comau S.p.A is a global supplier of industrial automation systems and services and is based in Turin, Italy. The company provides manufacturing systems for the automotive, aerospace, steel and petrochemical industries.
• Systems Sunlight S.A. is headquartered in Athens, Greece, and produces energy storage and power systems for industrial, advanced technology and consumer applications.

In the next post, we’ll look at activities at the sites and how the project is applying Augmented Reality at the different production facilities.

AREA member 3D Studio Blomberg (3DS) excels at visualization of data and especially at enterprise solutions for Augmented Reality. The AREA asked Pontus Blomberg, founder and CEO of 3DS, about his company’s history and projects in the space.

Q. Where do you have the greatest number of projects or customers?

Our customers are mainly in heavy industry, and include both large and mid-sized companies. We are also targeting the educational and consumer sectors for our AR solutions.

Q. How did 3DS become popular as a supplier to the industries you just identified?

Since the company’s founding we’ve led the way to digital transformation through advanced content delivery systems to promote process efficiency, expert knowledge and overall quality.

In 2006 we recognized the potential of AR to boost productivity in industrial workplaces and introduced the technology to Wartsila, a major Finnish power equipment supplier in 2008. At that time we evaluated ALVAR, Vuforia, and Metaio to survey their functionality from a visualization standpoint and assess their capabilities in handling 3D scenes and animations. In 2012 we delivered a proof of concept to Wartsila, and in 2013 we joined a Finnish national R&D program to study the potential of AR in knowledge sharing solutions for field service personnel.

This study showed that research and practical industry applications were not in sync, and many players were concerned with achieving efficiency through dynamic AR content and data integration. We entered an AR solutions provider partnership with Metaio in 2014 but realized the platform focused on technology functionality rather than on system utilization and process implementation, which is our focus today. We are currently studying the potential of Osterhaut Design Group’s R-7 smart glasses and continue to perform proof of concept projects with emphasis on process analysis, system development and AR in production use.

Q. What are the most common metrics used to assess task performance or project success?

We recommend that customer metrics be in line with their quality management system for effective reference and comparison. Broadly speaking, examples of common metrics include:

• Improvements in product and service quality
• Effectiveness
• Safety and risk reduction

Taking simple definitions of effectiveness (“doing the right thing”) and efficiency (“doing the thing right”), we believe it’s possible to work efficiently but it doesn’t contribute to productivity until we’re able to efficiently do the right things at the right time.

Q. What is your approach to AR introduction at customer sites?

As AR is new to most organizations, we recommend detailed analysis of the customer’s business strategy. In order to achieve digital transformation in line with the AR solution, the project needs to be aligned with the business strategy all the way to the board room. We also recommend demos and proof of concept projects to help organizations gain knowledge and understanding.

Q. How is data prepared for your customer projects?

It’s all a question of knowledge and experience gained through project implementation. Initially data has to be prepared manually, but at later stages of the project we’re better able to develop ways of handling new types of content in existing enterprise content systems.

Q. Do you get involved in the design of content that goes into pilot projects?

Yes, this is where our long experience and advantage really shines. Our expertise in visualization, combined with the customer’s industrial product and process expertise, play a significant role in achieving digital transformation through AR solutions. But no large-scale transformations can occur before new knowledge and tools are in place that allow for productivity and dynamic content.

Q. Do you study project risks with the customer or project leader?

There have been no major studies until now but naturally new technologies bring risks with them. Imagine driving your car with GPS assistance in heavy traffic and suddenly you can’t get a signal.

Q. Do you know if your customers perform user studies prior to and following use of the proposed system?

Yes, the fact that we start to see significant achievements in implementing AR solutions drives these kinds of studies. We’ve also had the chance to work together with partners in bigger collaborative research projects.

Q. What are the attitudes of those in the workplace where AR projects are successfully introduced?

Employees at the customer site are very positive and even surprised. We often encounter statements similar to, “Wow! I’ve seen this on YouTube and the Internet. It’s incredible to see that it really works.”

Q. Describe the technologies at play. What types of components do you offer?

Through our key partner network we offer the entire pipeline of smart glasses, mobile solutions, UIs, server-client databases and content development.

We use worldclass tracking technologies today but expect that Simultaneous Localization and Mapping (SLAM) technologies will gain ground. We realize this type of technology isn’t applicable in unique or dynamic situations at larger scales, although we’ve performed several demos and proof of concept projects with SLAM and the results are promising.

At the moment we see marker-based (or with code/ID) and geo-tracking as the most stable and flexible ways to acquire user context. We’ve built upon these technologies in our products and platforms.

At the same time we realize significant investment is needed in the modification of existing customer processes and new competences. To be successful, we aim to help our customers drive this change through systematic long-term cooperation.

Q. What must customers provide in terms of system components?

For rapid familiarization with the technology we recommend providing data to achieve a real look and feel. We recommend not overdoing it with complex UIs and information flows. Developing proof of concept projects with small, incremental steps for easy evaluation and quick changes is important to identify precisely the drivers of an AR introduction.

Q. With whom do you partner most often?

We partner with technology providers (hardware, software and tracking technologies), and we also see content providers as strategic because of their long-term customer relationships. To get all these complex systems to work together with business process changes is a team effort. It will take a few years. We aim to use what’s already been applied in an enterprise because we want to leverage the significant investments that have already been made in IT and visualization.

Q. What are the environmental conditions where customer projects are being conducted?

We’ve experienced both laboratory and real environmental conditions, especially in terms of lighting, vibrations and sound. Many of our customers use ruggedized solutions for their projects, which means unique and custom solutions for harsh, dynamic environments.

Q. What are your other offerings?

In terms of training, 3DS also provides competence development in combination with process development. For data, we use the customer’s cloud and offer commercial cloud solutions.

Q. What are the greatest challenges you currently face in AR introduction projects?

Customers often don’t have sufficient insight into the possibilities that emerging visualization technologies and content can provide. Therefore a clear understanding of customer expectations, goals and their business is needed. Customers also need a certain amount of trust that their expectations will be met.

Many times the only way forward is to agree on a proof of concept or demo that shows the technology, content, functionality, added value and supplier capabilities.

From the customer point of view, there are also uncertainties about the new types of content that will be needed to enrich the current PLM process to allow for visualization on a large scale. How will this information be connected and utilized together with the new visual content? We offer expertise in these questions and they need to be processed in very close cooperation with the customer as they touch the very core of their business.

Q. What are the future plans or next steps for your company?

We’ll continue to systematically monitor and build our international client base and partner network and develop state-of-the-art products and services.

If you saw the headlines coming out of Unity’s Vision Summit, you probably noticed a trend: Virtual Reality was the star of Vision Summit 2016. Valve’s Gabe Newell gave everyone an HTC Vive Pre. The Oculus Rift will come with a four-month Unity license. Unity is getting native support for Google Cardboard. At the summit, the expo floor had long lines for the “big three” head-mounted displays (HMDs): Sony’s PlayStation VR, Oculus Rift and HTC Vive.

It’s not that Augmented Reality was absent from what was billed as “The Definitive Event for Innovators in VR/AR,” but rather that the technology was in the minority of tools. This is the year of Virtual Reality, with the big three VR providers launching major products in March (Oculus), April (HTC) and sometime in the fall (Sony). The event was hosted by Unity, which caters almost exclusively to game developers needing comprehensive cross-platform development tools, and gaming in VR is expected to be huge. Virtual Reality was even the focus of the keynote, but astute observers might have noticed something.

Best Days of Augmented Reality Are Ahead

Unity’s own keynote referenced a report by Digi-Capital which predicts that the AR industry will have negligible revenue in 2016, but will surpass VR in 2019. In 2020, the AR industry is predicted to triple the value of VR. Take this with a grain of salt; Unity is in the business of selling licenses for their cross-platform game development toolset, so they’re incentivized to predict massive growth, but even reducing these numbers to a cynical level shows massive promise in a new field.

Most of this growth may be in gaming, but the AR presence on the expo floor leaned toward enterprise use. Epson was demonstrating their Moverio line of smart glasses, which has been around since 2012. Vuzix had their M-100 available to try, and they were eager to tout their upcoming AR3000 smartglasses.  In its booth, Vuforia demonstrated a Mixed Reality application on Gear VR that allowed the viewer to disassemble a motorcycle and view each part individually, which could be handy for vehicle technicians.

Of course, you can learn the most from hands-on experience with enterprise AR, which is exactly what NASA presented. They showed how they replaced complicated written procedures with contextual, relevant, clear instructions with AR using HoloLens. They also had a suite of visualization tools for collaborating on equipment design.

I presented the results of a year-long collaboration between Float and the CTTSO to develop an AR application designed to assist users in operational environments. We discussed the ins and outs of developing a “true AR” experience from the ground up, in addition to all of the lessons we learned doing image processing, using Project Tango, and more. At the end, I demonstrated the finished app, with integrated face recognition, text recognition, and navigation assistance supported either on an Epson Moverio or the Osterhout R-6.

An Increasing Focus

Vision Summit 2016 may have been a largely focused on VR, but that’s not a reflection of a lack of interest in AR. In our own research, we estimated that AR was lagging behind VR in terms of the technology readiness level by a few years. This was confirmed at the Vision Summit, but there’s still plenty of AR to get excited about. Valve even stated that they’d let developers access the external camera on the HTC Vive “in the long run” for Augmented and Mixed Reality applications. Expect next year’s Vision Summit to have a much larger focus on AR as this industry begins to truly take shape.

Did you attend Vision Summit 2016? What did you observe? Do you plan to attend the Unity event in 2017?