Interview with the AMRC: Augmented Reality in Advanced Manufacturing

Augmented Reality will play an important role in the future of manufacturing, but the details about where, how and who will benefit most are still unclear. The University of Sheffield Advanced Manufacturing Research Centre (AMRC) is an AREA member that, based on the organization’s collaborative research projects, is developing special insights on these topics.

As a faculty-level unit within the University of Sheffield, the AMRC partners with industry to conduct advanced machining and materials research that brings high ROI. The AMRC with Boeing focuses on manufacturing in aerospace, automotive and other high-value industries while, in parallel, the Nuclear AMRC focuses on manufacturing innovation and supply chain development for the civil nuclear and energy sectors.

We asked the AMRC’s digital manufacturing assembly specialist, Chris Freeman, to shed light on the drivers for introducing Augmented Reality in manufacturing and the risks he sees when Augmented Reality is integrated into projects underway or planned at the AMRC.

How do you identify where and how Augmented Reality can offer value?

We don’t propose any new technology unless we believe that it can address a specific operator requirement.  When our research partners come to us with manufacturing challenges, we make sure we study those problems in relation to not just the senior directive, but also the hands-on operator who performs the job day in and day out. They know the problems best and, often they provide the key metric, or problem around which the application needs to focus. For example, a process may need to be completed with fewer concessions or at a greater rate, but the operator will want to focus on how it will make their own life easier. That operator-level buy-in is crucial to having a successful deployment. Their personal experience in processes needs to be considered just as much as the goals of those senior to them. By presenting solutions that serve requirements of both operator and managers, benefits like traceability or the elimination of errors will more likely be realized.

We might recommend exploring AR as part of a solution when the key business challenge (or the opportunity to reduce costs) involves people interfacing with and using complex instructions or information in context.   

In each scenario we need to closely examine the whole process to help build a preliminary ROI model. We are always looking for robust business cases, where technology integration can deliver a step change because going to the effort to introduce something new for small incremental changes is not going to be justifiable. Even if it is innovative, the technology will not be adopted.

Existing data about operator performance is often not available but we may be able to collect indirect metrics or indicators of efficiency such as the number of rework orders or how much scrappage (waste) a company generates. The details may be extracted from the company’s manufacturing execution system (MES) or standard operating procedures (SOPs). These systems have the ability to gather a lot of detail for other purposes that we can use as part of a study to understand the business case.

Do you use or integrate with real time sensor networks or IoT in any part of your projects?

The vision of Industry 4.0 has always included a component of connected machines communicating with, and being controlled by, systems and humans in intuitive and low-risk fashion. We are being requested to do more projects with the Industrial Internet of Things but, at the moment, it’s still exploratory.

Augmented Reality is a great enabler for humans working with IoT but a great deal of potential value of IoT rests in the architecture and systems that sit behind it. Sensor networks are very powerful, especially when combined with real time Big Data analytics, and the use of Augmented Reality will enable new methods of data visualisation and human interaction.

How is data prepared for use in Augmented Reality experiences in an advanced manufacturing environment?

Ideally there’s no need to introduce a new data manipulation step between the source of the data and the AR experience user, but a lot depends on the use case requirements. We recommend that the AR experience system uses the raw data straight from its source, whenever possible, and not duplicate any existing functionality. We also recommend that the digital content be as close to its native format as possible.

The more complexity there is in data handling or mining, the less robust and less repeatable the process becomes. Translation and optimisation is often necessary but it’s not ideal. In situations where access to data sources is not available, any transformation processes need to be as automated as possible.

To determine how suitable a client’s existing data portfolio is we will always work with the customer’s existing data sources to prove out the process. Often there’s a learning process through which everyone goes, which with our help, allows them to understand what can be achieved with their existing data.

Risk management plays a large role in the process, too. When we’re planning an application, we’re always looking for the key challenges and risks. We examine potential issues and document these to ensure we understand the potential pitfalls.

What are the sources of risk (challenges) when using AR in the manufacturing environment and how do you address them?

With Augmented Reality there are many uncertainties about the technology itself, such as how it works in different environments. We have all sorts of challenges around natural and artificial lighting conditions, wireless network connectivity and many other factors that will impact user interactivity. Manufacturing environments have high levels of ambient noise due to industrial-scale machinery. This noise presents challenges with speech-based recognition interfaces. Selecting the right AR interaction mode for the right task is crucial.

We see rapid change in the features of hands-free displays. This raises uncertainty about how long one model will last before being superseded by a new one. Each change introduces new risks and costs. In order to lower the impacts of frequent model updates, it’s important to first implement a robust back end architecture.  Then, once that’s in place, the AR experience presentation hardware (wearables, tablets, phones, etc.) can be quickly removed and changed without the cost and delays of changing the underlying architecture.

In addition, there are risks associated with different recognition technologies. We have to evaluate image, bar code, natural feature recognition, SLAM and depth sensing with respect to the project goals and the environmental constraints. As integrators, we can also combine AR with well-established technologies such as geo-location sensing, RFID and Bluetooth.

Many of our partners are very security conscious and tightly regulated. Systems purely reliant on cloud-based architectures will not even be considered. Local networks are an option but still very much a problem when a number of organizations we work with don’t (or can’t) have Wi-Fi on the shop floor. This drives us to look at solutions that work entirely offline and then can connect with a data infrastructure after a shift or task is completed.

There are also project risks due to excessively high expectations. In other words, hype. We work with all stakeholders to make sure they are clear and realistic about their goals and match those to what the technology can do today.

What are people’s attitudes towards the adoption of AR and how do you manage those?

It varies highly. Most have done some basic user studies prior to beginning any investment in order to understand the potential for adoption of a new technology. However, the exact process investigated is unlikely to be specific to the one you’re working on. They will be keen to learn more but will want to see tangible metrics around value and ROI. At an operator level, they will probably have little awareness of the technology and so may be cautious about its use and how it will impact their day-to-day life. User engagement and trials are crucial in order to get buy-in at a shop floor level. The operators need to be involved to show that the new technology is helping the end user do their work. Then, the feeling is more open and likely to have positive support. If the presentation system is a hindrance in any way it will be discarded, hence the importance of engaging with all sections of the business.

We always work closely with our partners to educate both ourselves about their use cases, and them about any new technology components. Everyone must have an open mind about the opportunities that AR enables and, as we said, the many risks. We will continue to encourage our partners to use a progressive, value-driven approach to adoption of any new technology. And we look forward to working more with our AR technology provider network, including AREA members, to deliver solutions to address existing manufacturing challenges.