In a previous article, we introduced the University of Sheffield’s Advanced Manufacturing Research Centre (AMRC), a member of the AREA that develops innovative techniques and processes for high-precision manufacturing. A subsidiary, the AMRC with Boeing, collaborates with a variety of research partners in areas such as informatics, automation, robotics and Augmented and Virtual Reality. Besides aerospace, the results of this research into manufacturing are used in the automotive, construction and other high-value industries.

Earlier this year, the AMRC opened the doors of its newest manufacturing facility, Factory 2050, a glass-walled reconfigurable factory in Sheffield Business Park. The facility investigates and showcases new technologies and processes relating to Industry 4.0, including projects to explore digitally assisted assembly technologies to fill a looming skills gap in the aerospace industry.

Augmented Reality in Digitally Assisted Assembly

The Digitally Assisted Assembly (DAA) project focuses on techniques for delivering work instructions to factory operators, including the use of optical projection AR and wearables. According to the AMRC’s digital manufacturing specialist, Chris Freeman, the project allows partner companies to experience visual work instructions through a number of delivery mediums. Research includes:

• Optimizing AR tracking methods for effectively getting a part’s position to generate a frame of reference.
• Designing user experiences for work instructions that are projected or overlaid onto a part within the user’s field of view. These include instructions that guide users for tasks such as gluing sequences, fastener insertion, inspection, wiring looms, complex routines and more. The aim of this research is to reduce cognitive load and optimize the user experience for delivery across a variety delivery modes (e.g., projection AR) and devices from tablets to smart glasses.
• Using location-based services to add contextualized task and environmental information in relation to the user’s position or progress within a task.

With the technology still in its infancy, one of the aims of DAA is to simply demonstrate what can be achieved with the technology. Although smart glasses and wearables aren’t proven or certified for use in manufacturing, they are nevertheless being baselined for further research and possible future production usage. The AMRC are currently following a strategy of first identifying the “low-hanging fruit” from the current state of hardware and software, which means that research associates want to find some of the most obvious and perhaps least expensive options up front.

Skype for HoloLens

Although the AMRC is studying a variety of smart glasses brands such as ODG and Vuzix, remote collaboration use cases with Skype for HoloLens is an interesting application for meeting the needs of certification processes. This use case includes methods for lineside support and remote verification to complement or replace expensive quality management activities requiring the presence of a supervisor. It may even include assistance by remote colleagues when assembly or repair problems are encountered.

Freeman notes that though such use cases aren’t spectacularly advanced in terms of tracking in comparison with other scenarios such as overlaying geometric 3D models on objects being assembled, they are nevertheless disruptive of current manufacturing practices.

Projecting Work Instructions on Large-Volume Objects

Projected Augmented Reality, sometimes referred to as “spatial Augmented Reality,” features one or more optical projectors projecting a beam of light onto a specially designed work surface or even on the parts being assembled. Thus work instructions are displayed directly on surfaces to guide operators. The DAA is currently researching methods for effectively using projection AR in conjunction with both fixtures and robotic arms in work cells.

For example, an operator assembles aircraft parts with the assistance of robots to present a part at a better angle than if it were lying on a work surface. A robotic arm can swivel or position the part as needed by the operator, and projected AR is able to guide operators through a series of specific manufacturing procedures.

Defining Success

As has been discussed in other industry contexts, return on investment on any new technology can be challenging to define (whether it’s for AR or any other). Typical ROI calculations seek to determine the amount of savings a project can bring and when that investment will pay off. In the case of AR, relevant questions include how to quantify the value of conceptualized data and geometries for its usage in performance metrics.

Further research into AR will eventually uncover such answers, but in the near term, human factors and ergonomic studies can also quantify the technology’s effectiveness. For example, the AMRC is currently conducting AR-related training scenarios to determine a variety of metrics such as memory retention and AR’s overall effectiveness, as well as usability and operator response.

Beyond Aerospace

Although research being conducted at Factory 2050 aims to advance the state of the art in aerospace manufacturing, many of the techniques and procedures derived by DAA and other projects will eventually be used in other industries, such as energy and construction. For example, assembly techniques for large-volume aerospace parts can also be applied to assembling prefabricated homes at a factory as part of modular building manufacture. Having recently opened its doors, it’s apparent that the new facilities of Factory 2050 will have an impact on both present and future manufacturing in multiple domains for many years to come.

There are many use cases for Augmented Reality in the aerospace industry and the leaders in this industry have a long history with the technology. In this post, we review some of the milestones and provide highlights of the recent AREA webinar.

In 1969, while working in the Human Engineering Division of the Armstrong Aerospace Medical Research Laboratory (USAF), Wright-Patterson AFB, Thomas Furness presented a paper entitled “Helmet-Mounted Displays and their Aerospace Applications” to attendees of the National Aerospace Electronics Conference.

Over 20 years later the paper was one of eight references cited by two Boeing engineers, Thomas Caudell and David Mizell. In their 1992 paper published in the Proceedings of the Twenty-Fifth Hawaii International Conference on System Sciences, Caudell and Mizell coined the term “Augmented Reality.” The degree to which the team drew from the work of Furness, who had started the Human Interface Technology Lab at University of Washington in 1989, is unclear but the focus of the Boeing team was on reducing errors when building wire harnesses for use in aircraft and other manual manufacturing tasks in aerospace.

While the technology was not sufficiently mature to leave the lab or to deliver on its potential at the time, they suggested that with an AR-assisted system an engineer would in the future be able to perform tasks more quickly and with fewer errors.

Proof of Concepts

Approximately fifteen years later, in 2008, Paul Davies, a research & development engineer at AREA member Boeing began working with Boeing Technical Fellow, Anthony Majoros. Together, Davies and Majoros picked up where the Caudell and Mizell paper left off. They used commercially-available technologies such as Total Immersion’s D’Fusion platform to show how technicians building satellites could perform complex tasks with Augmented Reality running on tablets.

Airbus has also been experimenting with Augmented Reality for over a decade. In this paper published in the ISMAR 2006 proceedings, Dominik Willers explains how Augmented Reality was being studied for assembly and service tasks but judged too immature for introduction into production environments. The paper, authored in collaboration with the Technical University of Munich, focused on the need for advances in tracking.

Since those proof of concept projects, AR technology has advanced to the point that it is being explored for an increasing number of use cases in the aerospace industry. In parallel with the expansion of use cases, the pace of applied research into AR-enabling technology components has not abated.

Augmented Reality in Aerospace in 2016

While today AR may not be found in many aerospace production environments, the promise of the technology to increase efficiency is widely acknowledged.

On February 18, David Doral of AERTEC Solutions, Jim Novack of Talent Swarm, and Raul Alarcon of the European Space Agency joined Paul Davies and me to discuss the status of Augmented Reality in their companies and client projects.

Each participant described the use cases and drivers for Augmented Reality adoption. For Boeing, the key metrics are reduction of errors and time to task completion. Use cases include training and work assistance. AERTEC Solutions, which works closely with Airbus, and Talent Swarm are both focusing on use cases where live video from a head-mounted camera can bring greater understanding of a technician’s context and questions, and permit more rapid analysis and resolution of issues.

The European Space Agency sees a variety of use cases on Earth and in space. Inspection and quality assurance, for example, could benefit from the use of Augmented Reality-assisted systems.

Turbulence Ahead 

During the discussion, webinar panelists explored the obstacles that continue to prevent full-scale adoption. In general, most barriers to adoption can be considered as technological in nature. But there are also significant obstacles stemming from human factors and business considerations. We also discussed the degree to which other industries may be able to apply lessons learned from aerospace.

To learn more about the state of AR in the aerospace industry, please watch the webinar archive.

Do you have use cases and projects that you would like to share with the AREA and our audiences? Please let us know in the comments of this post.

Shipbuilding has been the perfect environment for industrial innovation for hundreds of years. Sails to steam, wood to iron, rivets to welds, blueprints to CAD, stick-built to modular construction–all major innovations to building extraordinarily complex vehicles. At Newport News Shipbuilding, we constantly seek new innovations to improve our safety, quality, cost, and schedules. Since 2007, we have explored Augmented Reality as a means to shift away from paper-based documentation in our work.

Since we began looking into AR for construction, operation, and maintenance workflows, we’ve come up with hundreds of use-cases to improve tasks or processes. These range from assisting shipbuilders in painting, ship-fitting, electrical installation, pipefitting, and more in several ways – on new construction ships, ship overhaul, facility maintenance, and decommissioning. Every use-case improves our ability to deliver nuclear aircraft carriers and submarines, but at different degrees of improvement.

We’re always adding new use-cases to the list, and we’ve needed to devise an adaptable framework for organizing and categorizing existing, proven uses and prioritizing future, potential use-cases.

Genesis of a Use Case

Augmented Reality should be employed first in places where it creates the most value – and that actually can be subjective. Sometimes, this is helping people become more efficient and working more quickly, sometimes this is about helping to reduce errors and rework, and sometimes it is all about improving safety. At Newport News Shipbuilding, a dedicated team of AR professionals help determine where AR is best suited, whether the technology is ready for the use-case, and how to best implement and scale a solution.

The first step in defining a use-case is performed by an AR industrial engineer, who determines where AR brings value in a workflow. She first meets with a skilled craftsman, and understands their challenges and needs. The industrial engineer identifies pain points in processes, such as when and where shipbuilders must consult paper documentation to complete a task. She must also consider human factors and always balance the needs of the craftsman against the capability of the AR solution as it can be delivered today.

Then, the AR engineer works with an AR designer and an AR developer to deliver a product. The AR designer determines the available data, components, interfaces and models for the system to satisfy requirements. Once the use-case is fully defined and the data is assembled, an AR developer implements software solutions, tests the system, and ensures reliable and adaptable development tools. At the end of the process, a new use-case is addressed, and a high-value product is delivered to the skilled craftsman.

A Classification Scheme

Over the years we’ve devised hundreds of use-cases and needed a way to understand and prioritize them. We started by categorizing them into a taxonomy that we think of as general, but we admit they might be specific to our business. We call these our seven use-case categories.

These 7 categories then are applied across three additional axes. These variables create a volume of exploration, or “trade space” for each use-case. The three application axes are as follows.

Return on Investment

When investing in new technology, it’s important to find those areas offering the highest return on investment (ROI) for every dollar spent. At the same time, there are potentially high value use-cases that are simply not conducive to an AR solution today. As a professional AR team, we pride ourselves on understanding when we can have an impact, when we can have a really big impact, and when AR technology simply isn’t yet up to the challenge. We primarily focus on advancing the seven use-case categories, and use the three variable axes to ensure we are maximizing customer value and ROI. As our expertise has grown, and as the technology matures, we have steadily increased value and readiness of AR throughout the entire trade space.

Today, we assess highest potential ROI and use that as a metric for scaling priority. Our model shows the greatest ROI in use-cases for inspection, work instruction, and training. Our focus there is now on scalability. We also know that the ROI is really tied directly to the technology readiness levels (TRL) of AR for those use-cases. While we are certain there will be benefit, maybe even higher ROI, on workflow management, operations, safety, and logistics – the readiness levels of AR for those use-cases within our trade space simply isn’t as high (today) as for the first three mentioned. You can’t scale what doesn’t yet work. So for the latter four uses, therefore, the investment isn’t in scalability, but rather in improving the TRL.

As Augmented Reality technology becomes more capable and less expensive to implement, enterprises will find ever-increasing uses. We’d like to learn how others in different industries have been developing theirs. Please share your comments and experiences with us.

The Human Factor in AR ROI

AREA research consistently shows that successful ROI analysis relies on implementations that involve the end user through the entire process. From defining and measuring the use case, during pilot testing, and through final implementation, effective stakeholder engagement generates significant benefits to the ROI process.

Identify and Engage Stakeholders Early

Think beyond the C-suite. Include IT, operations, HR, and end users. Each group has unique concerns and priorities:

• Executives: Focus on financial impact and strategic alignment
• IT: Address integration, security, and support requirements
• Operations: Highlight process improvements and efficiency gains
• End Users: Emphasize ease of use, training, and day-to-day benefits

AREA best practices emphasize involving stakeholders early, especially those who will use the technology daily. This approach maximizes user buy-in, ensures integrity of time and motion studies, and uncovers potentially unforeseen costs associated with different work environments.

Tailor Your Message with Data and Stories

Combine hard numbers from the AREA ROI Calculator with real-world examples. The AREA case study demonstrates how one company achieved remarkable results:

• 45% reduction in mean-time-to-repair
• 20% reduction in customer churn (from 2.5% to 2%)
• 90% reduction in parts inventory
• 25% reduction in audit costs

Use these concrete examples alongside your own projections to build credibility with different stakeholder groups.

Best Practices for Cross-Team Collaboration

AREA research identifies several critical success factors:

Establish Clear Metrics: Target specific business outcomes using Key Performance Indicators (KPIs). Common metrics for MRO applications include Mean Time to Failure (MTTF), Mean Time to Repair (MTTR), and Overall Equipment Effectiveness (OEE).

Ensure Financial Rigor: Collaborate with finance teams before pilots to ensure buy-in on business problems and measurement metrics. This cross-team collaboration is essential for ROI analysis that stands up to scrutiny.

Manage Change Effectively: Involve end users throughout the process to minimize “human” costs related to ongoing process change during deployment. Give users the ability to provide input on solutions, from hardware comfort to application value.

Assign a Champion: Organizations committed to maximizing ROI assign a “Champion” with sound grasp of both business and technology challenges. These individuals ensure projected ROI is realized after deployment and manage issues that could impact cost estimates.

Scaling Best Practices

For organizations moving beyond pilots to enterprise-wide deployments, AREA research recommends:

• Evaluate ROI on each pilot using the same framework
• Explore environmental factors that may differ from the pilot
• Standardize your approach to business case development
• Create rules-based frameworks for integration cost allocation

Leveraging AREA Resources

The AREA community provides extensive resources for stakeholder engagement:

• Case studies and best practice documents
• ROI Calculator with detailed instructions
• Research reports on specific use cases and industries
• Community insights and benchmarks
• Templates and frameworks for standardized analysis

Conclusion

With the right tools, data, and stakeholder engagement, you can turn AR from a “nice-to-have” into a strategic advantage. ROI is your bridge between innovation and impact—use it wisely, and leverage the proven methodologies developed by the AREA community to ensure success.

As AREA research concludes: “Understanding the potential ROI and following best practices is important for enabling the broader development of the enterprise AR ecosystem and driving AR solutions into the mainstream”.

Back to Infographic

Factors Distinguishing AR Device Vulnerabilities

The largest distinguishing factor in the inclusion of AR devices is the potential vulnerabilities that the devices can introduce to the enterprise. Similar to the impact potential associated with Industrial Control Systems (ICS) devices, AR devices allow bridges to data in physical space. In the case of ICS devices, the manipulation of the ICS is the objective, and the IT infrastructure is the pathway to the objective. AR devices however, allow unique perception of the environmental space in which the device exists by virtue of its external sensing capabilities. Audio, Video, Spatial Mapping, Thermal, and Geographic Location features all collect data that can be used or captured for adverse purposes. These data sets could allow capture of data that would allow an attacker to circumvent a protection mechanism or allow the transition of attack from one vector to another. This change in the attack chain, or vector transition consists of exploiting a network connected to the AR device, capturing the data and using it to attack a non-connected system. Examples include password eavesdropping on an air-gapped system, alarm system PIN capture, token capture, or user behavior tracking.

In examining the security factors associated with AR devices in the enterprise it is important to identify the factors that differentiate the devices from traditional IT systems. Similarities in architecture, OS and communication protocol will allow for like attack vectors from a vulnerability perspective. AR devices have a very specific set of impacts that deal with the unique attributes that AR devices bring to the enterprise.

AR devices by their inherent feature sets provide distinguishing impact characteristics in two areas:

1) As a collector of environmental data

2) As an injector of data to a user

The first differentiator is most significant as the AR devices inherently allow collection of visual, audio, network configuration, user behavior, and environmental behavior along with normal data consumption. These unique elements in the exploit chain allow for significant new methods and impacts as a result of a cyber attack. As the weakest point in any cyber defense is always the human element, these devices can significantly increase the ability to exploit human interface mechanisms and physical protection mechanisms.

Attack Categories for the AR Device

In determining and communicating threats and attack vectors it is advisable to provide a common nomenclature and identification schema. This will allow end-customers and penetration testing groups to address specific threat areas of interest derived from specific customer concerns normally described in narrative format. These attack categories can be used both to frame the scenario and synchronize elements of concern with specific threats, exploits and vulnerabilities.

Data Capture