Today, companies in many industries seek to develop AR and VR applications for their needs, with the band of existing Augmented Reality solutions extending from gimmicky marketing solutions to B2B software. Helping production companies train their workers on the job by augmenting service steps onto broken machines is one of those solutions.

Augmented Reality could assist designers or architects to see a product while it is still in development. It could facilitate a marketing and sales process, because customers can already “try on” a product from a digital catalog. Or it could assist warehouse systems so that users get support in the picking and sorting process

The list of opportunities is endless and new use cases are constantly arising. The whole point of using AR is to make processes easier and faster. While at first, Augmented Reality and devices like smart glasses seemed way too futuristic, new use cases make them increasingly suitable for everyday life in the workplace.

Recognizing Objects and Characters

Augmented Reality is based on a vital capability: object recognition. For a human being, recognizing a multitude of different objects is not a challenge. Even if the objects are partially obstructed from their view they can still be identified. But for machines and devices this can still be a challenge. For Augmented Reality this is crucial though.

A smartphone or smart glasses can’t display augmented overlays without recognizing the object first. If needed for correct augmentation, the device has to be aware of its surroundings and adapt its display in real time according to each situation, all the while changing the device’s camera viewing angle. Augmented Reality applications use object detection and recognition to determine the relevant information needing to be added to the display. They also use object tracking technologies to continually track an object’s movements rather than redetecting it. That way the object remains in the frame of reference even if the device is moved around.

Character recognition is also crucial for a device’s understanding of the environment, as it not only needs to recognize objects, but according to the use case, it might also have to “read” it. This provides an even better discernment of the types of information that are important to process.

Optical Character Recognition

Optical Character Recognition (OCR) deals with the problem of recognizing optically processed characters, such as those in the featured image above. Both handwritten and printed characters may be recognized and converted into computer readable text. Any kind of serial number or code consisting of numbers and letters can be transformed into digital output. Put in a very simplified way, the image taken will be preprocessed and the characters extracted and recognized. Many current applications, especially in the field of automation and manufacturing, use this technology.

What OCR doesn’t take into account is the actual nature of the object being scanned. It simply “looks” at the text that should be converted. Putting together Augmented Reality and OCR therefore provides new opportunities; not only is the object itself recognized, but so is the text printed on that object. This boosts the amount of information about the environment gathered by the device, and increases the decision-support capabilities offered to users.

The Potential of OCR

Data import still requires high processor power and camera resolution and is expensive. Nevertheless OCR offers a viable alternative to voice recognition or input via typing.

Using OCR with smart glasses offers improvements for different kinds of business processes. Imagine a warehouse worker who needs both hands free to do his job efficiently. Using smart glasses to overlay virtual information on his environment can make him more efficient. But the ability to automatically scan codes printed on objects just by glancing at them frees his hands for other tasks.

Another example would be the automation of meter reading. When a device identifies the meter hanging on a wall, as well as its shape and size, and then automatically scans its values, a greater amount of meters can be read per day. This use case could be useful to energy providers.

When you look around, you will realize how many numbers, letters and codes need to be either written down or typed into a system every single day. Such processes, which can be very error prone, can become much less painful using OCR.

Due to the potential to radically change user engagement, Augmented Reality has received considerable and growing attention in recent months. Pokémon Go certainly has helped and, in turn, generated many expectations for the advancement of AR-based solutions. In fact, the game has provided the industry with a long overdue injection of mass appeal and as a result, significant investment from (and among) tech giants around the world.

From corner shops to large utility providers, the spike in popularity of this technology has everyone buzzing about how it could improve their business. The flexibility of implementation, from improving processes to stand-out marketing solutions, has also altered the expectations of these prospective clients as they seek personalized enterprise-level AR-based solutions. Consequently, the time has come for vendors and suppliers to consider a new model when it comes to managing customer expectations.

When deploying Augmented Reality solutions in an enterprise context, it is essential to build strong partnerships with your customers, and in many cases to take on the role of a trusted advisor. This becomes more important through the stages of delivering a project—starting with defining a proof of concept (POC) to implement bleeding edge solutions with operational teams and ultimately end users, who in fact are the actual users of the technology.

While the primary value of Augmented Reality systems is to allow for the contextual overlay of information to enable better decision making, the visual data overlay and various data sources and devices that trigger location sensors all come into play—converging in the form of a complex mesh. Vendors must note that partnerships are key to solving the pieces of this puzzle.

Service Delivery—Creating Value from the Complex Mesh

This complex mesh is what ultimately garners value as the assimilation of these technologies creates new and innovative social and business ecosystems and associated processes. When addressing enterprise adaptation, one must be aware of the following questions:

• How best can value be driven into workable solutions in an enterprise?
• How well does it integrate with existing legacy systems?
• Would new skills be required to introduce and manage the change?
• Does the solution deliver increased productivity or efficiencies, i.e., better utilization of resources or allow for better decision making through information?
• Does the solution enable new revenue models for the organization that are consistent with the existing product and service offerings?
• In turn, how does this solution affect the profitability of the organization?
• Last, but not least, is the business rationale clear for the implementation of such a solution?

The move towards customer-centric systems means that your customer (or your customer’s customer) is at the center of all decision making. This may be a shift from their existing system practices, meaning it’s even more critical that the chosen change management process be well aligned to the client’s corporate culture.

The Client’s Point of View—Questions to Ask When Going Beyond the POC

Some of the questions that vendors need to consider when it comes to implementing the solution beyond the POC are:

• What is changing?
• Why are we making the change?
• Who will be impacted by the change?
• How will they react to the change?
• What can we do to proactively identify and mitigate their resistance to the change?
• Will the solution introduce new business or revenue models?

Working as one with your customers through innovations to operations is a key factor for success. The complex mesh of AR, VR, IoT and Big Data technologies makes this even more critical as enterprises see an integration of their digital content, systems and processes.

It is essential to take a partnership mindset—where the Augmented Reality innovation solution is built both for and with the customer, and through a customer-implemented change management process—to quickly and easily create ROI as well as tangible, actionable outcomes.

Since its release on July 6th, Pokémon Go has become a global phenomenon—with downloads of the mobile app exceeding 75 million within the first three weeks. Many reviewers credit the game’s meteoric success to its innovative use of “Augmented Reality.”

To those of us in the enterprise AR community, of course, Pokémon Go is no more augmented reality than Atari’s 1972 “Pong” arcade game was table tennis. In Pokémon Go, the merging of the virtual and real worlds is confined to the projection of 2D monsters on real-life backgrounds. While it’s a novel effect for a mobile game, it barely scratches the surface of what Augmented Reality can do—or its tremendous potential for enterprises to achieve greater operational efficiencies.

Still, we at the AREA have to view the success of Pokémon Go as an important milestone in the development and adoption of AR for the following reasons:

• Pokémon Go is familiarizing the world with the basic concept of Augmented Rreality. Hopefully our members can spend less time having to explain what Augmented Reality is—or how it differs from Virtual Reality. Potential customers will already understand the basic concepts and be ready to learn more.
• Pokémon Go is proving that AR is no longer a futuristic concept. If an AR game is already a commercial success, can widespread enterprise AR solutions be far behind? Companies that had previously taken a wait-and-see approach to AR may now be more motivated to explore the possibilities for their businesses.
• Pokémon Go is proving that people are engaged and excited by the technology. The game makes it vividly clear that AR is a powerful and compelling tool people enjoy using. That enthusiasm can only help fuel the growth and development of the AR market.

Many media outlets and bloggers agree and are driving the conversation in our direction. Just look at some of these recent headlines:

• “Is Pokémon Go Really Augmented Reality?”
• “How Pokémon Go Took Augmented Reality Mainstream”
• “Why Pokémon Go is a Game Changer for Augmented Reality and Marketers”
• “Pokémon Go is Nice, But Here’s What *Real* Augmented Reality Will Look Like”

Our challenge now is to leverage the Pokémon Go phenomenon to accelerate the adoption of AR in the enterprise. That means taking the opportunity—as the AREA members Gaia Dempsey of DAQRI and Scott Montgomerie of Scope AR have done recently—to make sure inquiring media outlets understand that the impact of enterprise AR will be even more significant and lasting than the current Pokémon Go craze.

To find out more about the AREA contact Mark Sage, Executive Director.

Further to my last post about the AWE ’16 conference, I want to share some thoughts and areas for future focus from my first two months as the Executive Director of the AREA.

It’s exciting to be involved in such a vibrant and dynamic ecosystem of AR providers, customers and research institutions. I’m amazed at the sheer breadth of the kinds of member organizations, their offerings, skills, achievements and desire to work with the AREA to help achieve our shared mission of enabling greater operational efficiencies through smooth introduction and widespread adoption of interoperable AR-enabled enterprise systems.

Success and Challenges

Through my initial conversations with the members, I’ve learned of many success stories and also the challenges of working in a relatively young and rapidly changing industry.

For example, AREA members talk about the prototypes they’re delivering with the support of software, hardware and service providers. However, I would like to see more examples of wider rollouts, beyond the prototype stage, which will encourage more buying organizations to investigate AR and understand its massive potential.

The AWE conference in Santa Clara in June, and the subsequent AREA Members Meeting added emphasis to my initial thoughts. The AR in Enterprise track of AWE, sponsored by the AREA, highlighted a number of organizations who are already using AR to create real benefits, ranging from the enabling of real time compliance, better use of resources, applying the most relevant data and the reduction of time, errors and costs. It was great to see that many member companies understand the benefit of working together to enable the whole AR ecosystem to become successful.

Carrying on the Momentum

My continued focus over the coming weeks will be to carry on the great momentum that has been started. I’m briefing more organizations from all over the world about the benefits of becoming an AREA member. I’ll continue the focus on developing and curating thought leadership content including case studies, frameworks and uses cases, and deliver them via the AREA website, webinars and social media. We’re enhancing our value proposition through our development of research committees that increase the capabilities of the industry.

This is an exciting time for the enterprise AR industry and the AREA; I’m very interested in any feedback or comments you may have so please contact me at [email protected]. I look forward to hearing from you and working with our growing membership to meet our goals of realizing the potential of Augmented Reality in the workplace.

AREA member JoinPad provides cloud-based and contextually aware software that simplifies processes in a number of industries. The company’s BrainPad product integrates enterprise resource systems and sensor networks to add Augmented Reality visualization and contextual computing to existing business processes.

This month we interview Nicolas Pezzarossa, Global Sales and Business Development Director of JoinPad, about the enterprise use cases his company is encountering for its products and services.

In which industries are you finding the greatest interest for your products and services?

We see strong interest from providers of energy supplies and infrastructure. Oil and gas has the largest proportion of such companies. Besides this, we’re finding companies in other industries getting involved with Augmented Reality:

• Energy
• Automotive
• Manufacturing
• IT hardware, infrastructure and services
• Retail
• Tourism

We’ve also provided solutions for use cases in these industries.

What are the reasons for AR’s popularity in these companies?

We believe it’s due to growing awareness of the value that Augmented Reality brings in conjunction with digital transformation. The ROI of individual AR use cases is becoming evident, and there’s an increasing maturity of hardware platforms for this environment.

We’ve also received much interest in our smart glasses SDK, and as well in our Smart Assistance solution that offers guided assistance as an “augmented operator’s manual” and expert collaboration in real time.

With whom do you partner most often?

We partner with well-consolidated players active in the field of consulting to large industrial companies in IT infrastructure for manufacturing processes, where we can supply the AR-related components in an OEM-type of integration.

Has employee performance in the workplace prior to AR introduction been studied by your customers?

Most of our customers have detailed statistics about performance or time taken to complete specific tasks, and to which we can correlate our solution. In other cases we’ve performed a detailed analysis of their work processes. Our product also contains a module for work order management that enables generation of KPIs for specifically measuring this type of work performance for comparison purposes.

What are common metrics, and do you recommend customers choose their own?

We find that in most cases the most important factor is time to complete a task (for increasing efficiency). But others include the ratio of possible to actual mistakes and the value of avoided damage, as well as the level of fatigue or satisfaction of operators.

As we are discussing the consequences of a disruptive technology, another important factor is the possibility of enabling new work processes. Although this is more difficult to measure, it offers large potential for increasing efficiency.

We always emphasize the importance of evidence for a return on investment in all phases of a project. This is also essential for advocating internally to stakeholders and management for the adoption of AR.

What is your company’s recommended approach to introducing AR in an organization? Are there steps or a model or method you follow?

We take a phased approach and in a preparatory phase offer a workshop for defining possible use cases and analyzing current work processes. We then propose a proof-of-concept phase in which we offer a basic solution with limited functionality. This allows the customer to experience the new solution and see its potential. We subsequently initiate a pilot phase with actual data exchange, followed by a roll-out phase where the application is introduced into actual work processes.

How is data prepared for your customer projects?

All data must be processed to efficiently support the use case. In particular, when connecting to an ERP system it’s important to choose the data sets specifically supporting the use cases.

Do you get involved in the design of the content that will be used in pilot projects?

Normally the customer asks us to provide the content as well as the design of the user interface. In the case of smart glasses this can involve an innovative interaction design. Key to project success is to propose visualizations that help solve the specific problem at hand and improves visual perception.

Our experiences working with customers have allowed us to develop specific templates for smart glasses applications that ensure efficient intake of the relevant information.

What is the profile of the typical person who performs the selected tasks prior to AR, and what are their attitudes?

Augmented Reality, particularly when used with smart glasses, has the major benefit that even untrained operators can perform complex tasks. But also highly trained operators benefit from availability of real time data where it matters.

In most cases operators are satisfied about working with innovative tools that they appreciate as supporting their work tasks. But the impact of new technologies on human resources and work safety must nonetheless be carefully monitored.

Do you study project risks, and do customers perform user studies?

Risk analysis is always part of our use case analysis, just like recommended fallback scenarios.

Although most customers don’t plan user studies themselves, we offer a questionnaire process both before and after a pilot for evaluating improvements for purposes of the roll-out phase.

What are the system components the customer must provide for a successful project?

This is highly dependent on the use case but there is in fact no requirement that customers provide us with system components. However at various times they do provide us with components ranging from full packages of 3D files to databases and API access.

What type of recognition and tracking technologies do you support, and what are the effects of lighting?

We work with all recognition and tracking principles (e.g., image, bar code, natural features, SLAM, depth sensing, etc.), but based on our proprietary core algorithms.

Lighting represents a challenge that in many cases can be overcome, yet it influences tracking stability. It’s always possible to correct this influence using other types of sensors, or to reduce its impact with fallback scenarios.

Do you use IoT, and is AR content locally archived or accessed over a network?

We have specifically developed and deployed an IoT module in our AR platform BrainPad that is used today by one of our customers in the energy industry to retrieve data from sensors on industrial equipment in the field in real time. We thus fully support IoT data integration.

For AR content, there are different scenarios involving both kinds of access and integration, depending on the workflow.

What are the greatest challenges you face in current projects?

One of the largest challenges is in the need to prove ROI on every single use case, which is often complex as many industrial and manufacturing processes are highly intertwined with other processes.

What are the future plans or next steps for JoinPad?

The next steps are to further grow our activity and supply more publishable customer use cases to further support the adoption of the technology in industry. In particular, Joinpad will intensify its education effort to spread knowledge about the value and design of AR applications by conducting workshops offered to technology experts and managers, as well as in academic initiative.

Our team at 3D Studio Blomberg, along with key partners, travelled to Santa Clara, California, to attend the Augmented World Expo. The event is the largest annual conference and exhibition about Augmented Reality worldwide, with over 4000 attendees and 250 exhibitor booths. During the two days, I had the opportunity to make interesting new contacts, meet other AREA members, see and try a variety of innovative AR and VR solutions and attend the enterprise AR tracks hosted by the AREA.

Larger Players Entering the Market

Judging by the offerings on display at AWE, the ecosystem for enterprise AR products and services is expanding. Players like PTC (Vuforia), Osterhout Design Group (ODG) and Microsoft through HoloLens had observably increased their footprint at the event, and even the presence of VR products at an AR show confirmed the overall trend of a growing ecosystem. Microsoft presented its HoloLens product hosted by Vuforia and its technical capabilities are impressive. We view all this as a positive development as it will bring increased competition and more innovative market offerings.

AR in Enterprise Sessions

The AREA-hosted AR in enterprise track featured speakers and AREA members on a diverse range of topics from IoT to security. The sessions were interesting but they highlighted the array of challenges still facing companies seeking to implement Augmented Realty in the workplace. One fundamental takeaway was that widespread adoption of AR in industry isn’t solely a technological issue of AR, but rather is the result of steady improvements in the surrounding mix of technologies such as IoT, Big Data, etc. As these enabling features and technologies improve, they make the value proposition of AR even more compelling.

Another insight from the sessions was the idea of mental models and how we imagine innovations should work—but that actually turn out to be quite different in reality. We need to avoid this pitfall when thinking about AR and the problems it solves.

Lastly, partnerships are essential for expanding the ecosystem and assuring its success. For example, ODG makes great smart glasses but they need partners that create virtual content in order to get the most out of their products. All of these key ingredients will produce the necessary lifting power to make AR a killer app.

Conclusion

AWE was a rich, rewarding experience that we and our partners in attendance enjoyed immensely. As content providers for AR-enabled enterprise systems, we appreciated the opportunity to meet a variety of potential partners to which we add value. We’re looking forward to turning the ideas gained from the conference into reality, and to contributing to the exciting and growing marketplace for Augmented Reality.

As the incoming executive director of the AR for Enterprise Alliance, I was very excited to attend my first Augmented World Expo and to meet some of the 34 members of the AREA.

AWE is one of the largest and best-attended events worldwide about Augmented Reality, and typically hosts thousands of attendees and hundreds of companies. This year’s event was no exception and did not disappoint. I was pleased to meet a high number of innovative AR companies from the AREA provider segment and attend demos of their groundbreaking solutions. It’s clear to me that AR in enterprise is here to stay and the AREA occupies a strategic position in growing the entire ecosystem to the benefit of everyone.

Benefits of AR in Enterprise

The event gave me the opportunity to speak with a range of attendees from many companies and markets. It was exciting to be asked so many different and interesting questions on many topics and one conclusion that came up time and again was the importance of AR in enterprise. The potential benefits and savings of AR is getting the attention of C-suite rather than just the innovation and technology teams. The trajectory towards a real reduction in time, costs and errors is a critical for companies as they look to streamline their business and increase the return on investment.

Enterprise AR Track at AWE ‘16

The focus on enterprise was supported by an impressive number of customers and providers presenting their experiences during the Enterprise AR track—sponsored by the AREA. I learned a lot from all the presentations but it was also instructive to listen to the members of the AREA’s customer segment. They were insightful and provided a unique perspective on the benefits and issues they experienced when implementing AR solutions within their companies. It’s clear that there are many lessons to learn and the AREA is well placed to help the AR ecosystem make effective and informed decisions based on shared knowledge and experience.

The AREA at AWE

At AWE we experienced a constant stream of people visiting our stand and asking questions. Many expressed appreciation of the AREA’s work and benefits achieved for the ecosystem. A number of them even mentioned regularly visiting the AREA website when trying to find information about AR, and that the AREA’s content was insightful and informative.

For those who hadn’t heard of us, it was useful to discuss our mission, benefits, membership options and growth. Much interest was expressed and I hope new members will join based on these discussions.

AWE was my first real experience meeting the enterprise AR community and it was a very useful and insightful experience. I look forward to following up with the many attendees I met and help drive the AREA’s development and its role in supporting this nascent ecosystem.

AREA Members Meeting

After AWE, we held an AREA Members Meeting in Palo Alto, California, on June 3. It was an honor to chair my first such meeting. AREA in-person meetings occur around three times a year and they’re a great opportunity to meet with members, discuss progress made, define future strategic plans to further develop the ecosystem and have some fun.

Thanks to Atheer for hosting the event at the beautiful Palo Alto Art Center.

The morning agenda items included:

• Progress updates from the various AREA committees
• Upcoming events in which the AREA can support its members

The afternoon included various brainstorming sessions around the content and the way the AREA positions itself to potential new members.

The day was full of insightful and interesting discussions, and from a personal perspective it was great to interact with many leaders and understand how we can work together as an alliance to support and grow the ecosystem and provide thought leadership to possible new customers and providers of AR.

If you are interested in joining the AREA, please complete this form.

This year’s Laval Virtual conference showcased innovations in Augmented Reality and introduced wide-ranging discussions on the topic of Augmented Realty in enterprise. On the second day, AREA board member Christine Perey hosted a round table session on the use of Augmented Reality to promote productivity. Participants included Manuel Asselot (Robocortex), Sebastian Knoedel (DIOTA), Marie-Julie Pecoult (Diginext), Pontus Blomberg (3D Studio Blomberg), Yann Froger (EON Reality) and Jim Novack (Talent Swarm).

Olivier Larroque of Capgemini provided his impressions and summarized the essential questions and answers discussed by the invited panelists in a post on the blog of RA’Pro, an AREA member. The original post in French is translated and provided in English here.

Which industry currently leads the way with Augmented Reality in Europe?

Among the panel participants there was agreement that aerospace is a leading industry. Adoption of AR in aerospace is driven by reuse of 3D content in its product life cycle management (PLM). AR use cases are most recommended, for example, where they can overcome a technician’s lack of experience in performing a task, or where they can assist in risky or complex operations. Such use cases are being applied in a highly regulated environment where only one error on the assembly line can cause the loss of an aircraft or satellite.

What are characteristics of use cases that are most compelling for investors?

Ideally, you should identify critical points of a business process where human errors generate the greatest cost. The return on investment (ROI) on AR as reported by companies such as Boeing and Newport News Shipbuilding shows how crucial it is for an enterprise to embark on such a project.

Boeing conducted a comparison of three different guidance methods on a satellite assembly procedure of 50 steps using instructions:

• On a stationary PC
• On a table (PDF)
• Overlaid in the field of view (AR) using a tablet

The results were dramatic: during first-time assembly, the AR-enhanced tablet users with no prior experience with the steps committed one error, while those using the PC committed eight errors. During the second time following the steps, those using AR committed no errors.

Which prerequisites should be in place before a company implements AR?

Optimization of existing 3D content for Augmented Reality is actually more important than products or toolchains. Maintaining a flexible and modular approach in adopting these new technologies provides the ability to move among varying hardware and software products and packages, and helps organizations to remain a step ahead of the market.

What do you think of smart glasses?

The first step in adopting Augmented Reality is to use an AR-enhanced tablet for testing and then migrate to smart glasses if appropriate.

Smart glasses, of course, have compelling features (hands-free working, portability, etc.), but one should avoid falling into “shiny object syndrome,” or the desire to adopt technology at any price without first examining all its ins and outs. It’s essential to study what’s actually required, as well as the technical limitations.

What are the implications of adopting AR as a disruptive innovation?

The implications are threefold:

• Social resistance to change: unions and conservative individuals within the company may be reluctant to change if AR is seen as an aspect of robotics. Communication should be oriented to assisting humans and what they do best. Moreover people, the employees, should be included at the heart of the discussions.
• Enhancing procedures for Augmented Reality: start with simple tasks in workbooks or manuals. AR takes advantage of our visual processing and operators tend to instinctively apply it to minimize errors as it superimposes instructions to be followed in real time.
• Dealing with technological realities: over-the-top special effects in concept videos have instilled high expectations for AR. We should rather address and educate the customer about the technology’s limitations in terms of hardware (field of view, tracking, etc.) and software. The technology should be thoroughly tested to ensure it matches the use cases that the customer is targeting.

If the best hardware is not currently available, when is the best time to get started with AR?

Ideally right away. AR adoption is a long process with many different aspects (social, technological, security, etc.) that involves deep collaboration among all domains of an enterprise (operational, management, legal, etc.).

Conclusion

Augmented Reality will allow companies to approach the way their employees work more visually, with a new way of representing objects, learning new tasks and transmitting knowledge. Recall the progress made between the first MS-DOS screen and what we have today with personal computers. With new devices such as smart glasses and other products like Microsoft HoloLens, the DAQRI Smart Helmet or Magic Leap, the changes will also be very dramatic.

A hackathon is an event to help groups of people with different skills collaborate intensely over one or two days towards a shared goal. Hackathons bring together people who are passionate about technology with those who have challenges that technology can solve. One of the affiliate AREA members, the AEC Hackathon, organizes such events to address architecture, construction and engineering challenges. Teams of AEC hackers have been gathering in cities around the world under this banner since late 2013.

Hackathons can also be a valuable way to raise awareness and identify new requirements. The goals of the first AR Standards Hackathon, organized by the WEKIT European Project, were to explore the AR standards landscape and attract engineers who are interested in the adoption of existing and emerging AR standards.  The hackathon was hosted by RWTH Aachen University in Germany on April 15-17, 2016.

Kick Off

This was actually my first hackathon although I’ve been using standards and supporting their development through the IEEE Standards Association and meetings of the Community for Open and Interoperable AR. When I arrived for the kick-off on Friday evening, participants were gathered around the registration table. Soon the organizing team (Prof. Ralph Klamma, István Koren and Peter De Lange) welcomed participants and everyone got acquainted. I discovered eager people with many ideas and a high level of enthusiasm, and generally find those with AR experience development skills to be very rare.

The hackathon kicked off with Dr. Fridolin Wild introducing us to the WEKIT project and schedule. Then the organizing team presented the sponsors and explained their donations in support of the event’s goals:

• Vuzix provided their M100 smart glasses
• Atheer provided copies of AiR, their AR software development platform
• Optinvent provided their ORA2 smart glasses
• Epson provided their BT-200 smart glasses
• Bitstars was represented by their HoloBuilder authoring platform

Those with an idea for a project pitched it to the participants in an effort to recruit team members. Some of the proposed projects were very compelling, for example, using AR to help a deaf person to understand what a speaker is saying just by looking at the speaker through smart glasses. Another project proposed to reduce the impact of graffiti by helping artists to express themselves on walls without leaving any trace in the real world. There were also several Augmented Reality games.

The Projects

On Saturday morning, the teams formed and the hackathon officially started. For the next 36 hours, nearly 30 people were busy with coding, brainstorming and troubleshooting.  

Five teams worked on projects that included:

1. #hackWalls : collaborative creation of digital AR-activated artworks and notes on physical walls.
2. #faceglARss : social interaction and people identification using face recognition and AR emotional status tags.
3. #cyclops : two-way interaction in room-sized exercise games based on fiducial markers.
4. #boxingAR : a gesture-tracking experience in boxing-related exergames.
5. #explorAR : architectural presentation and interactive browsing of 3D models using Leap Motion for customer demos.

At the end of the second day, the participants presented their projects while the jury gave them scores based on:

• Originality of the project
• Use of originally developed modules
• Use of open source solutions and standard-based technologies
• The possibility to further develop and expand the project in different contexts

Participants were tired but proud of their projects and achievements.

Results

The #hackWalls team won the award for best project. The creative team developed a web browser-based solution that allows anyone to draw on any wall and to share it with anyone else with a simple URL.

Hackathons are a great opportunity to meet new, like-minded people that have passion in AR and development skills. The event permitted me to hear and sharing ideas, build useful contacts and I am already looking forward to next AR hackathon!

Have you attended a hackathon? Is there a hackathon in your future?

Augmented Reality will play an important role in the future of manufacturing, but the details about where, how and who will benefit most are still unclear. The University of Sheffield Advanced Manufacturing Research Centre (AMRC) is an AREA member that, based on the organization’s collaborative research projects, is developing special insights on these topics.

As a faculty-level unit within the University of Sheffield, the AMRC partners with industry to conduct advanced machining and materials research that brings high ROI. The AMRC with Boeing focuses on manufacturing in aerospace, automotive and other high-value industries while, in parallel, the Nuclear AMRC focuses on manufacturing innovation and supply chain development for the civil nuclear and energy sectors.

We asked the AMRC’s digital manufacturing assembly specialist, Chris Freeman, to shed light on the drivers for introducing Augmented Reality in manufacturing and the risks he sees when Augmented Reality is integrated into projects underway or planned at the AMRC.

How do you identify where and how Augmented Reality can offer value?

We don’t propose any new technology unless we believe that it can address a specific operator requirement.  When our research partners come to us with manufacturing challenges, we make sure we study those problems in relation to not just the senior directive, but also the hands-on operator who performs the job day in and day out. They know the problems best and, often they provide the key metric, or problem around which the application needs to focus. For example, a process may need to be completed with fewer concessions or at a greater rate, but the operator will want to focus on how it will make their own life easier. That operator-level buy-in is crucial to having a successful deployment. Their personal experience in processes needs to be considered just as much as the goals of those senior to them. By presenting solutions that serve requirements of both operator and managers, benefits like traceability or the elimination of errors will more likely be realized.

We might recommend exploring AR as part of a solution when the key business challenge (or the opportunity to reduce costs) involves people interfacing with and using complex instructions or information in context.   

In each scenario we need to closely examine the whole process to help build a preliminary ROI model. We are always looking for robust business cases, where technology integration can deliver a step change because going to the effort to introduce something new for small incremental changes is not going to be justifiable. Even if it is innovative, the technology will not be adopted.

Existing data about operator performance is often not available but we may be able to collect indirect metrics or indicators of efficiency such as the number of rework orders or how much scrappage (waste) a company generates. The details may be extracted from the company’s manufacturing execution system (MES) or standard operating procedures (SOPs). These systems have the ability to gather a lot of detail for other purposes that we can use as part of a study to understand the business case.

Do you use or integrate with real time sensor networks or IoT in any part of your projects?

The vision of Industry 4.0 has always included a component of connected machines communicating with, and being controlled by, systems and humans in intuitive and low-risk fashion. We are being requested to do more projects with the Industrial Internet of Things but, at the moment, it’s still exploratory.

Augmented Reality is a great enabler for humans working with IoT but a great deal of potential value of IoT rests in the architecture and systems that sit behind it. Sensor networks are very powerful, especially when combined with real time Big Data analytics, and the use of Augmented Reality will enable new methods of data visualisation and human interaction.

How is data prepared for use in Augmented Reality experiences in an advanced manufacturing environment?

Ideally there’s no need to introduce a new data manipulation step between the source of the data and the AR experience user, but a lot depends on the use case requirements. We recommend that the AR experience system uses the raw data straight from its source, whenever possible, and not duplicate any existing functionality. We also recommend that the digital content be as close to its native format as possible.

The more complexity there is in data handling or mining, the less robust and less repeatable the process becomes. Translation and optimisation is often necessary but it’s not ideal. In situations where access to data sources is not available, any transformation processes need to be as automated as possible.

To determine how suitable a client’s existing data portfolio is we will always work with the customer’s existing data sources to prove out the process. Often there’s a learning process through which everyone goes, which with our help, allows them to understand what can be achieved with their existing data.

Risk management plays a large role in the process, too. When we’re planning an application, we’re always looking for the key challenges and risks. We examine potential issues and document these to ensure we understand the potential pitfalls.

What are the sources of risk (challenges) when using AR in the manufacturing environment and how do you address them?

With Augmented Reality there are many uncertainties about the technology itself, such as how it works in different environments. We have all sorts of challenges around natural and artificial lighting conditions, wireless network connectivity and many other factors that will impact user interactivity. Manufacturing environments have high levels of ambient noise due to industrial-scale machinery. This noise presents challenges with speech-based recognition interfaces. Selecting the right AR interaction mode for the right task is crucial.

We see rapid change in the features of hands-free displays. This raises uncertainty about how long one model will last before being superseded by a new one. Each change introduces new risks and costs. In order to lower the impacts of frequent model updates, it’s important to first implement a robust back end architecture.  Then, once that’s in place, the AR experience presentation hardware (wearables, tablets, phones, etc.) can be quickly removed and changed without the cost and delays of changing the underlying architecture.

In addition, there are risks associated with different recognition technologies. We have to evaluate image, bar code, natural feature recognition, SLAM and depth sensing with respect to the project goals and the environmental constraints. As integrators, we can also combine AR with well-established technologies such as geo-location sensing, RFID and Bluetooth.

Many of our partners are very security conscious and tightly regulated. Systems purely reliant on cloud-based architectures will not even be considered. Local networks are an option but still very much a problem when a number of organizations we work with don’t (or can’t) have Wi-Fi on the shop floor. This drives us to look at solutions that work entirely offline and then can connect with a data infrastructure after a shift or task is completed.

There are also project risks due to excessively high expectations. In other words, hype. We work with all stakeholders to make sure they are clear and realistic about their goals and match those to what the technology can do today.

What are people’s attitudes towards the adoption of AR and how do you manage those?

It varies highly. Most have done some basic user studies prior to beginning any investment in order to understand the potential for adoption of a new technology. However, the exact process investigated is unlikely to be specific to the one you’re working on. They will be keen to learn more but will want to see tangible metrics around value and ROI. At an operator level, they will probably have little awareness of the technology and so may be cautious about its use and how it will impact their day-to-day life. User engagement and trials are crucial in order to get buy-in at a shop floor level. The operators need to be involved to show that the new technology is helping the end user do their work. Then, the feeling is more open and likely to have positive support. If the presentation system is a hindrance in any way it will be discarded, hence the importance of engaging with all sections of the business.

We always work closely with our partners to educate both ourselves about their use cases, and them about any new technology components. Everyone must have an open mind about the opportunities that AR enables and, as we said, the many risks. We will continue to encourage our partners to use a progressive, value-driven approach to adoption of any new technology. And we look forward to working more with our AR technology provider network, including AREA members, to deliver solutions to address existing manufacturing challenges.