Which Aviation Groups are Providing AR and VR Solutions?

Celebi Aviation Holding are setting up an aviation academy in Turkey. The Celebi Aviation Academy in Turkey being certified by the International Air Transport Association (IATA) and Training Validation Program (TVP). Recognising the academy as an official Center of Excellence in Training and Development. Allowing for the student to virtually sit airside, along many Airbus and Boeing planes. Covering various elements of commercial aviation, from pre arrival too post departure inspection. The various scenarios presented give aviation students a safe environment Letting students find solutions to identified faults. Allowing for difficult environments. Such as different environmental conditions, under LVO (low visibility operations) and night time operations. Japan Airlines teaming with Asia’s largest manufacturer Tecknotrov and Quatar Airlines to invest in more autonomous training for pilots and engineers. Expectations for AR and VR in aviation training has always had a strong relation; both recently and throughout previous generations of the technology. So, it comes as no surprise that, with the well respected foundation between the mixed reality space and aviation. The predicted growth rate of the AR and VR aviation markets is expected to be more than $1372 million by the end 2025. It is important to note the practical strides is where AR and VR fit into the expected processes for a workers day to day routine. With solutions for almost every team member involved in the flyers journey. AR offers flight attendants and handlers a paperless workflow, obviously aiding with cross contamination in post pandemic, busy work environment. SATS, the chief ground-handling and in-flight catering service provider at Singapore Changi Airport. Having integrated M300 smart glasses to 600 of their employees. Getting rid of pen and paper methods during luggage handling. Allowing for quick QR scanning, saving a reported 15 minutes for each flight.

What Can Passengers Expect?

Passengers are also become part of this landscape; VR can offer flyers new forms of entertainment during their long journey. Airfrance are partnered with SkyLights. A VR inflight entertainment group working from San Fransico. Together they have created a unique headset for Airbus A340 flights. Skylights boast a massive success rate with passengers using their VR entertainment headsets during flights. With a 90% recommendation rate and 4h average usage time among passengers. Lufthansa are also innovating for their passengers. Creating a 360-degree immersive experience for passengers to watch while travelling. With worldwide prospects for flyers and aviation workers, when flyers return to airports in mass. They could be presented with more AR and VR options than ever. Making the return to the runway a breeze.

Manifest, AR Remote Guidance

The AR technology firm provided direct onboarding during a trial with British Airways, which introduced Taqtile’s products to onsite workers.

Dirck Schou, the CEO of Taqtile, added,

“This unique accelerator program has been a great way to introduce airlines to cutting-edge technologies like Manifest which can help them improve the performance of technicians and engineers immediately”

Additionally, the 10-week programme taught staff how to utilize the Manifest platform. Taqtile’s service operates on spatial devices such as Magic Leap and Microsft’s Hololens 2; the product also works across multiple devices, including tablets and smartphones.

An onsite worker can access guidance resources such as digital manuals, video guides, holograms, and 3D models, all presented as detailed AR visualizations. Manifest displays digital resources in the field of view (FoV) of a worker’s headset, and the wearer can navigate a spatial interface hands-free.

Schou continued, stating,

“Through demonstrations of our AR-enabled work instruction platform over the 10-week program, airline industry leaders have gained a better understanding of the tangible benefits Manifest is capable of delivering”

For frontline airport staff, Taqtile’s solution helps workers learn invaluable company-centric knowledge and enhance their efficiency when performing maintenance tasks.

Taqtile explained how airports could leverage its Manifest solution by dispersed teams providing live guidance from an operations centre to frontline employees.

Manifest supports several file types, including photos, videos, real-time 3D (RT3D) content, computer-aided designs, and PDFs. Taqtile recently teamed up with Microsoft this month to integrate the Azure Remote Rendering platform into Manifest.

The move enables firms to perform large-scale onboarding, training, and operational duties with increased efficiency and engagement. Taqtile and Microsoft achieve this by integrating Azure-powered streaming to enhance RT3D content distribution across Manifest-ready devices.

Key updates in this release:

• Advanced Model Target Improvements: Training times for Advanced Model Targets have been optimized and now depend on the number and size of views. Recognition performance for advanced, close-up views has also been improved.
• Area Target Improvements:
  • The target’s occlusion mesh is now exposed in the C API which allows native apps to render occluded virtual content in combination with Area Targets as you move through the space.
  • Textured authoring models are now created by the Area Target Creator app and the Area Target Capture API providing an improved authoring experience in Unity. These scans can be loaded into the Area Target Generator for clean-up and post-processing.
  • Area Target tracking data is now compressed and takes up to 60% less space.

• Unity Area Target Clipping: Area Target previews in the Unity Editor can be clipped based on height, for faster previewing and better visibility of the scanned space during app development.
• Engine Developer Portal (EDP) Self-Service OAuth UI: OAuth Engine credentials can now be managed from the EDP, eliminating the need for the command line.
• Notices
  • High Sensor-Rate Permission: Due to new Android permission requirements, developers should add the “high sensor rate” permission to all native Vuforia Engine apps running on Android 12+ for all releases, otherwise VISLAM may not work. Read more about VISLAM tracking here.
  • Permission Handling: The Vuforia Engine behavior of triggering OS-specific user permission requests at runtime is deprecated in 10.8 and will be removed in an upcoming release.  All native apps should be updated to manage permissions themselves. The 10.8 sample apps share best practices for this.

Due to Covid-19, restricted travel and social distancing requirements have accelerated AR-based solutions for remote assistance, since it has now become a necessity for industry operations. As opposed to video-calling applications such as Zoom or Skype, Augmented Reality is software technology allowing users to overlay graphic material onto video images via mobile network. For instance, an expert and on-site technician could both view a panel of switches on an annotatable screen connected via their own PC, phone, or tablet. The expert could then circle or drop a virtual arrow on the part that needs attention for the technician to see on the screen.

Key advantages of AR-based remote assistance noted in the article include:

• Instantaneous feedback system – users interact with the elements of their work in addition to with one another
• Highly mobile form of communication – includes annotations, on-site images, and graphic augmentation as well as a two-way voice connection
• Increases safety – e.g. reduces number of healthcare workers needed in a hospital room, limiting exposure to Covid-19
• Helps maintain workflow continuity
• More intermittent, special-purpose vehicle of communication than video conferencing

Other potential features of AR, dependent on the specific software, include:

• Retaining images
• Issuing push notifications
• Looping in multiple users
• Recording a session for future training
• Object character recognition
• Transferring files

The article concludes by stating that, even post-pandemic, AR is likely to become a prominent tool for operating personnel and frontline technicians to rely on.

Projection Augmented Reality, sometimes also referred to as “spatial Augmented Reality,” is a method of delivering digital information to users within a stationary context. Target objects and users can move around in the environment, but the zone in which AR experiences take place is limited to the fields of view of both the fixed projector and supporting camera for tracking.

The first example of projection Augmented Reality was called the “Digital Desk.” In November 1991, within months of their contemporaries at Boeing, Tom Caudell and David Mizell, coining the term “Augmented Reality,” William Newman and Pierre Wellner, then researchers at University of Cambridge and Xerox EuroPARC, published a paper in the UIST 1991 conference proceedings called the DigitalDesk Calculator: Tangible Manipulation on a desktop display.

In this 8 minute video, Pierre Wellner explains the concept and demonstrates the working prototype of the first system.

How it Works

Projection Augmented Reality features one or more optical devices (projectors) that project a beam of light onto a specially designed work surface and in some cases directly on the parts on which a user is working. This provides immediate guidance for tasks and reduces the need to interrupt workflows to consult information elsewhere.

Workspaces for projection Augmented Reality also feature any of a variety of stationary cameras. Cameras are positioned to track objects with or without fiducials. Control of the workspace environment, such as lighting, reduce the computational complexity of the tracking algorithms.

Once configured, the projection Augmented Reality system can provide user instructions or assistance in a variety of media. For example, digital information can be:

• Text, for example, cycle time count down
• Images, for example, blueprints or simple directional arrows
• Animations
• Videos

Some systems also provide assistance by way of task-synchronized audio.

Benefits of Projection Augmented Reality

Projection Augmented Reality can offer the following benefits:

• Reduces or eliminates the need for computer monitors and screens, as the instructions appear directly in the task space.
• Reduces users’ cognitive load when following work instructions due to the fact that there is no need for “attention switching” between work instructions and the task at hand.
• Integrates into manual workflows by promoting a “no faults forward” policy to ensure and confirm correct execution of the preceding step.
• Provides feedback on completed tasks for process improvement, traceability and unique digital IDs for build cycles.

Use Cases

Projection Augmented Reality can optimize performance of some types of production and logistics tasks when the work can be performed at a station (rather than the user going to the workplace or moving around in a larger space).

Tasks that can benefit from projection Augmented Reality include:

• Assembly
• Disassembly
• Inspection
• Part knitting
• Sequencing
• Maintenance
• Tool changeovers
• Gauging
• Welding
• Wire harness routing
• Glue bead replacement
• Logistics (shipping and receiving)
• Inspection
• Training

Have you asked any IT professionals or business managers what they’re doing with Augmented Reality? A small fraction can share how they’ve considered using AR for improving their workplace processes, but most inquiries about how companies are using AR begin with a blank stare and end in frustration.  

The AREA and its members are developing high-quality content that can be the basis of more precise and fruitful dialog than we often have today. Once there is a shared conceptual foundation, we’ll be able to discuss the concrete benefits as well as the risks of introducing Augmented Reality in the enterprise with our audiences.

Explore the Audience Knowledge Level

Casual discussion between acquaintances or between a supplier and a potential customer can’t evolve gracefully if they must begin with deep explanations or clarifications of confusing terminologies. Don’t start with a dry definition. Focus first on either a known or shared challenge or potential benefit and make sure you can squeeze a few terms in casually in the first minutes.

“Isn’t it frustrating that we can’t significantly increase our productivity?” you can inquire. Be specific about the use case, if you can. You can substitute “increasing productivity” with other metrics such as reduce errors, reduce risk or increase safety. Drop in some keywords to make sure they understand that you feel new technologies could help. Avoid buzzwords such as wearables, IoT, Augmented Reality or Virtual Reality in the first five minutes. Try to avoid bringing up Hollywood movies or popular science fiction books that have Augmented Reality.

Then you can say that you’ve heard or that you’re exploring how this new technology could play a role by overlaying digital information on the real world. Let your prospective customer or partner, or whomever you’re speaking to, be the first to mention wearables or AR.

When asked if they’ve heard of it and what they’re doing or planning to do with Augmented Reality, an IT professional will respond in one of two ways. The younger the person, the more likely they are to have heard and understood the potential. That said, they may not have thought to apply it to their job.

“That’s technology for your smartphone. I’ve seen it used in a museum, once” they might say. Then they either describe how the AR experience failed or just didn’t bring value to them.  Such conversations often conclude with the person dismissing the whole idea.

“It’s probably good for entertainment, but we’re not that kind of company,” is not an uncommon conclusion.

A more knowledgeable audience may remember Virtual Reality and the promises it held but didn’t deliver. Then you will need to reprogram them to understand the differences. 

Others will have had no exposure at all to Augmented Reality.

Light Bulb Moment

Once you’ve decided if the conversation is worthy of continuing investment, you’re going to aim for a “light bulb” moment: a look in their eye that shows that the person with whom you’re meeting has had a breakthrough in understanding.

To get to that moment of realization may take several steps. As already suggested, if you’re in conversation with an IT professional or line manager with a lot of engineering experience, you will get there more quickly.

Begin by building upon something very familiar. Everyone has seen and almost all have personally used video conferencing. AREA member David Doral, Director of AERTEC Solutions begins his education process by suggesting that when trying to understand a problem at a remote location, it would be valuable to be able to see things as if from another’s eyes.

“We suggest to the customer that we support the technician in the field or on the shop floor with an expert who is somewhere else,” explains Doral. He doesn’t say where that expert is, but makes it perfectly clear that they are the key to solving a problem and there’s not time for that expert to personally fly to the location. In AR, this use case is known as the “remote expert,” but this term doesn’t need to be introduced.

“Then, if they like this concept, we can suggest that the expert could draw arrows, point or otherwise indicate steps with animations,” continues Doral. “Imagine that the person who is in the field or on the shop floor is providing the remote hands, performing tasks as directed and under the supervision of the expert.”

Up Close and Personal

Another approach to reach a light bulb moment is to demonstrate an Augmented Reality experience right away. Sometimes, this can be performed using a tablet and an object that you’ve brought with you. Choose an object that is likely to be professional and slightly complex in nature but with a very simple user interface, such as a pocket projector. A virtual interface can appear with Augmented Reality to help the user with configuration and operation.

Three-dimensional objects are nice and have a big “wow” factor but a photo will also work well and may have higher performance. Lighting, and reflections on a glossy surface, may have a big impact on your ability to track the target, so test your sample photo or object well before using it. Be sure to give the other person the device to hold and move around, to interact with the content in the experience.

Often people try to simulate this effect, and reduce the risk of failure, by showing a video of an AR experience recording, but your audience will assign lower credibility to a video because they understand that special effects as seen in the movies are now commonplace.  Hasn’t everyone seen Minority Report and Iron Man?

From a shared understanding of the benefits of Augmented Reality, you might be able to progress to talking about a project and the potential of implementing AR in a few use cases.

What techniques have you used to successfully start a conversation about enterprise Augmented Reality?  Share your methods with others in the comments below.

Which of the many use cases for enterprise Augmented Reality should you implement first?

Selecting the best use cases for enterprise Augmented Reality introduction is arguably one of the most important steps that business managers will perform when exploring the technology’s potential to impact workplace performance.

During the ARise ’15 conference, Carl Byers, president of the AREA and Chief Strategy Officer of Contextere, presented key concepts and provided valuable recommendations for those who are planning to introduce AR in their organizations. This post builds upon those remarks.

Why Use Case Selection is Important

Careful selection of use cases for your company’s first AR project is critical for several reasons. First, the project will be used to choose the tools and to pilot the selected technologies while learning their benefits and limitations. Second, successful results will illustrate AR’s potential and help obtain buy-in for further investments from other groups and management.

The enterprise IT department is frequently involved with vendor selection and assessments of new tools. Since hardware is almost always involved in the delivery of AR experiences, the IT department may consider support for enterprise mobility management, connectivity and data security among other processes and objectives. The evaluation of a vendor’s training and support programs may also be performed during or in parallel with the development of the first project. Consider use cases that leverage prior positive experiences with IT introduction projects. A use case in a department that has not had prior IT-assisted technology introductions may introduce unforeseen problems.

Other departments, for example, human performance support and training organizations, also frequently feel they have a stake in how Augmented Reality is introduced.  Their interests need to be weighed and considered when selecting initial use cases.

Driving Internal Rate of Return for AR

The ultimate goal of introducing a new technology is to improve operational efficiency. Efficiency might be improved by driving down costs or time, or improve workforce productivity. Sometimes capturing the full value of a new technology involves organizational change.

When evaluating possible AR use cases, it’s important to consider how deeply changes associated with AR introduction may impact a business process, or multiple processes. An initial, low-cost research project in a sandboxed environment or an isolated field support improvement for one specific piece of equipment may be just right. But if the organization’s management is exploring more dramatic changes, AR introduction may be part of a larger initiative.

When considering the details of the AR introduction project and calculating IRR, it’s important to examine the productivity changes that could, once demonstrated for an isolated case, be applied across an entire factory or line of products and customers. Consider how a few small and specific pilots could meet your long term goals.

That said, it’s well known that large-scale change is usually slower. Should an AR pilot be considered as part of a larger organizational transition, the project may have to cope with many more variables and could experience greater delays.  The good news is that, if proven in the context of a broader change management approach, AR adoption may be driven from within as “just an integral part” of the organizational improvements.  

Complexity that’s Easily Tracked

Early resistance to AR projects has, in some organizations, been traced to the fact that the task or use case that was selected for AR testing was easy for an employee to perform unassisted. The lesson is that if there isn’t a pain point, AR isn’t needed.  Don’t waste valuable time, money or political “capital” of an organization.

That said, there are also risks in overreaching with respect to the current state of the art of Augmented Reality. If the user pain point proposed for a use case involves conditions that are difficult for current AR systems to identify or objects that are difficult to track, the technology may not perform reliably. Lack of reliability and repeatability fuels doubts and generally reduces the user and management’s appetite for the new technology.

Leveraging Existing Enterprise Data Stores

Developing the first AR experiences for a pilot requires new skills, methodologies and tools. Rather than adding to the project workload by developing new content as well, a use case can reuse or leverage existing enterprise data.

While some assets may need to be modified or adapted for mobile delivery platforms, overall project complexity will be lower and less costly when new AR experiences are based on existing enterprise data.

Involving Mission Critical Systems

Whenever mission critical or other high-impact enterprise systems are involved in an AR pilot project, the project may be escalated to management levels that are more risk averse: the CEO doesn’t want to do anything that might impact sales and stock prices, and the C-suite frequently shares that aversion to risk. On the other hand, if you can gain their support, their subordinates will be on board with the project and there will be fewer delays due to internal doubts.

The need for deep testing of any interface with a mission critical system, if that’s the route that’s recommended for an early AR project, is more costly and time consuming and may introduce unanticipated delays. If testing fails, integration with mission critical systems may cause the project to be cancelled.

Recommendations

1. Choose one or a few use cases where value can be measured clearly. For example, reduced down time, increased safety or compliance.
2. Focus on simple, practical, quick value capture. In the figure below we show how seven different factors can be weighted:
   1. Use a standard network architecture
   2. Design for bursty communication for longer battery life
   3. Identify where there’s large differences between novice and expert performance
   4. Make it easy to capture and repeat best practices
   5. Find use cases where some network services (e.g., videoconferencing with an expert) or special equipment (e.g., safety glasses) is already required
   6. Solve a current or recurring pain point
   7. Provide access to enterprise data systems via mainstream (legacy) interfaces
3. Thoroughly document all assumptions, steps taken and feedback, and share these with your technology partner.

Source: APX Labs

Choose and Choose Again

There are potentially hundreds of interesting use cases, and we’re currently building a use cases listing on the AREA site.

Choosing the initial AR use case is, as we’ve discussed, important but not the end of the process.

Frequently there are multiple AR use cases that can impact the operational efficiency of an enterprise. As a result, it’s not unusual for an AR pilot project to take workflows of multiple departments or processes into account. If this is the case, make sure the different use cases are well defined and the lessons learned in one project are captured and applied to others.

What are the initial use cases you’ve considered for evaluating whether Augmented Reality is right for your organization?

Want to hear more? Watch this video…