Critical infrastructures (CRITIS) such as the chemical storage terminal of Attilio Carmagnani are of central importance to our modern society. Very high demands are placed on their operational security as failures or impairments of CRITIS can have serious consequences. For example, terrorist or drone attacks on assets of the chemical sector could pose severe risks to public health and safety.

However, maintaining the operational safety of these infrastructures presents a challenge. Due to cyber-physical threats critical infrastructures are exposed to an increasing potential of danger and keeping up with security issues has become more difficult in the last years.

About EU Horizon 2020: InfraStress

The Horizon 2020 project InfraStress has taken on this challenge and aims to create protection in the form of innovative technology. A consortium of 27 partners from 11 countries started to work on developed methods to improve the cyber-physical security and developed an open testbed stress-testing system. Augmented reality training is one part of InfraStress’ set of measures.

“By using augmented reality and smart glasses, we work on a better assessment of these critical and industrial environments. Via AR operators and workers can be trained, how to clamp down on cyber-physical threats and hazards,” states Alexander Werlberger, CTO at Holo-Light. “The collaboration with our partners Attilio Carmagnani and STAM was very enriching: we learned a lot from each other and created a very customized solution for industrial requirements.”

Emergency Training with Augmented Reality

Situational awareness is crucial in case of emergency. Every action must be right. With AR personnel can be trained through real-time simulations. It prepares team members to make critical decisions on the spot and to have realistic experiences without endangering anyone.

Irene Bonetti, Terminal Director of Attilio Carmagnani affirms: “During training we have to imagine fire or liquids. With the AR training solution on HoloLens 2 we can really see it and understand how the chemicals will behave. Also, the fact that AR trainings can be executed one-by-one multiple times helps us to prepare in a better way.”

Conclusion

As governments and enterprises see the potential of improving disaster response skills with augmented reality, analysts expect a rise of AR training solutions in the public sector. Also, the increased roll-out of 5G networks will have a positive influence on its further spread. Ultra-high bandwidths and low latency as well as the increasing deployment and acceptance of cloud and edge computing will power digital transformation through XR (an umbrella term for all immersive technologies).

Qualcomm has unveiled its latest research milestones and innovations for advancing the next chapter of 5G, and they looks likely to transform industries worldwide. The company’s new and updated R&D over-the-air (OTA) testbeds and system simulations are enhancing the 5G system foundation to bring even more capacity, wider coverage, and lower latency to mobile operators and devices on a global scale. The testbeds and system simulations also highlight 5G’s horizontal capabilities to transform a wide range of industries from industrial IoT, automotive, enterprise, and more.

“Qualcomm Technologies is committed to driving the mobile ecosystem forward by breaking technology barriers and bringing superior 5G system performance to new industries, use cases and experiences for the smartphone and beyond,” said John Smee, vice president, engineering, Qualcomm Technologies, Inc. “Our new 5G R&D technology OTA testbeds and system simulations build on more than 30 years of deep wireless technology expertise, end-to-end system knowledge, global experience, and exemplify our leadership in advancing 5G for the next decade,” Smee continued.

Strengthening wide-area 5G system foundation

The latest wide-area 5G OTA test networks, prototypes, and system simulations announced by Qualcomm illustrate new capabilities and efficiencies of the 5G system. The latest demonstrations are focused across five key wide-area 5G research topics including subband half-duplex, wide-area 5G positioning, cross-node machine learning, 5G NR-Light evolution with sidelink, and 5G device disaggregation.

Expanding mobile mmWave deployments

New mmWave OTA prototypes and system simulations have been announced by Qualcomm, a pioneer in mmWave technology, and are exemplified in five key areas. These are smart mmWave repeaters, machine learning-based beam prediction, network topology optimisation, mmWave IoT expansion, and mmWave in smart factories.The company’s demos  showcase how 5G mmWave is evolving to support even better performance, deployment efficiency and expanded use.

Smarter transportation with 5G V2X sidelink

Qualcomm’s automotive vision for 5G V2X (vehicle-to-everything) is to bring increased road safety by connecting cars to other cars, pedestrians, bicyclists, road infrastructure and more. Their new 5G Rel-16 aligned R&D prototype further drives the 5G V2X sidelink evolution for delivering enhanced network capacity by offloading high-bandwidth data such as local high-res 3D maps from wide-area 5G networks, as well as bringing more robust communication with distributed time synchronisation to maintain communication even when GNSS is unavailable.

Enabling new Industry 4.0 efficiencies

The 5G over-the-air industrial test beds from Qualcomm illustrate key features pushing the boundaries of 5G technology, which sets the foundation for Industry 4.0. Advanced technologies showcased by Qualcomm are time sensitive networking (TSN) with micro-second level time synchronisation, ultra-high reliability with coordinated multi-point (CoMP), indoor precise positioning with centimetre level accuracy, and the novel introduction of sidelink to industrial IoT for more efficient device-to-device communication.

Boundless XR over 5G mobile mmWave

Using their optimised end-to-end 5G mmWave OTA system, Qualcomm showcases efficient simultaneous support of multiple virtual reality users across diverse applications, such as workplace collaboration and gaming. This will drive the convergence of physical, digital, and virtual worlds, boundless XR ushers in more immersive mobile virtual and augmented reality experiences through 5G and distributed computing.

Better system efficiencies for greener networks

With mobile networks growing to support wider bandwidths, denser deployments, and more users, Qualcomm’s researchers are also pursuing greener networks. This includes new advanced techniques, such as digital post distortion and high-efficiency modulation schemes that can greatly reduce the energy consumption of a mobile network system while sustaining high performance data links.

Qualcomm Technologies’ latest research shows the growth of new 5G phones, deployments, and applications across wireless technology, edge cloud processing, and AI. Indeed, the company’s new prototypes underscore Qualcomm’s commitment to ongoing research and development of future 5G technologies and beyond.

Read Qualcomm’s AREA

The goal of the 3D Thinking articles is to provide exercises, tools, technology, techniques to unlock a new dimension in your brain.

Because you’ve spent most of your life in front of 2D flat screens like computers, tablets, TV, or phones, it’s likely that your mind’s natural 3D thinning capabilities have been limited.

Today, we have 3D technologies that break 2D limitations, enabling us to create and communicate in 3D, leveraging the full spatial capabilities of our brain.

This is the second article in the series (you can read the introduction here).  In no time at all, you’ll be well on your way to creating a premium presentation and delivering results with AfterNow Prez:

The topics are:

1- Choosing an Augmented Reality or Virtual Reality Headset

2- Playing Games to Develop Your 3D Skills

3- Apply Your 3D Skills with These Fun VR Creator Tools

You can read the guide here. 

Read Afternow’s AREA member profile here. 

The digitization of processes in the logistics centers is an important factor for this. For Rexel Germany it was particularly useful to find new solutions for the regular in-house tours and maintenance. With Inspect AR from Microsoft partner Netcetera, these work processes can be made even more efficient on Microsoft HoloLens 2 in the future.

Support for employees with security checks

If you imagine a logistics center with its huge buildings and technical systems, it becomes clear how important special measures are here for fire protection and the safety of people and objects.

For this reason, employees at Rexel Germany previously had to use manually managed Excel lists to check at regular intervals whether the fire extinguishers were always in the right positions and ready for use. During such tours there is a risk that checkpoints on the list have been mixed up or left out. In addition, the employees needed time to orientate themselves in the warehouse.

With Inspect AR on HoloLens 2, employees are now guided to the right places in the logistics center by holograms. The Microsoft Azure Spatial Anchors technology locates each test point geographically in the building. At the examination points, employees also receive supporting information in the form of images, videos and texts and can confirm the points in real time on digital checklists.

In this way, the new digital solution with Inspect AR on HoloLens 2 increases the overall accuracy with which security tours are carried out. In addition, the mixed reality solution also simplifies the onboarding of new employees who do not yet know the building. You will be guided step by step through the logistics center by Inspect AR on HoloLens 2.

With HoloLens 2 and Inspect AR, everything is in the right place

Another challenge in the logistics center is to equip all workplaces with the necessary materials. For this purpose, regular tours are carried out, during which the employees check whether there are dispensers for adhesive film, printing equipment or similar materials at all workplaces.

In order for the colleagues to find their way around the desk, the materials should always be placed in the same position and functional. Various sample tables, which are compared with the workstations on checklists, serve as templates. In the previous work process, employees of Rexel Germany noted such steps with manually managed Excel lists.

Logistics employee carries HoloLens 2 in the logistics center

The new process with Inspect AR on HoloLens 2 simplifies this significantly: Employees put on HoloLens 2 and have their hands free for their tasks throughout the work process. At the same time, all the information required can be conveniently displayed in the field of vision.

In addition, there is no need to compare the printed checklists, because all information is digitally fed directly into the Inspect AR web management tool. This enables digital management of the entire process – from creating templates for work instructions and checklists, through creating and monitoring tasks, to viewing results reports.

“With Inspect AR and Mixed Reality, you can switch from paper to digital work instructions in just a few hours, instead of taking days or weeks. Our software makes the know-how of experts available virtually on the real object. This has the following advantages: Faster execution and more efficiency, clear documentation with photos, videos, text and a 100% transparent overview of all processes. ”- Elias Remele, Business Development Manager Netcetera

Thanks to the digital database of tours and maintenance with Inspect AR on HoloLens 2, Rexel Germany now quickly receives a comprehensive overview of work processes and can further optimize them based on this.

“Even greater security for our employees in the logistics center, reliable processes for our customers and a reduction in our CO2 footprint: with the introduction of mixed reality test processes in the logistics center, Rexel Germany is consistently taking the next big step towards digitization and sustainability.” – Nicole Steuer, company spokeswoman Rexel Germany.

Read Microsoft’s AREA member profile here 

Topics included are:

• What is Augmented Reality?
• The scientific classification of Augmented Reality
• How does Augmented Reality work?
• Hardware
• Software toolkit
• Software application

The market is expected to reach US$  303,934.14 thousand by 2028 from US$ 145,287.76 thousand in 2020

It is estimated to grow at a CAGR of 9.9% from 2020 to 2028

The report highlights the trends prevailing in the market, along with market drivers and deterrents.

Surgical smart glasses provide a smarter way to perform surgeries across the globe. Surgeons wear these glasses while performing a surgery and expert professionals from outside can view the surgery through these smart glasses and give advice regarding the surgery.

Surgeon in Operation Theater can communicate with another surgeon with the help of microphones of the glasses and share the surgical view with the wide lens camera on the surgical smart glass.

A few prominent players operating in the surgical smart glasses market are

• MEDITHINQ CO., LTD
• Taiwan Main Orthopaedic Biotechnology
• IRISTICK (AREA member)
• DTU Srl
• Vuzix Corporation (AREA member)
• Pixee Medical
• RODS AND CONES
• AMA XpertEye
• ImmersiveTouch, Inc
• Microsoft Corporation (AREA member)

Market players are launching new and innovative products and services to sustain their position in the surgical smart glasses market. For instance, in Jun 2020, MediThinQ commercially launched wireless smart glasses GV-200C. These wireless smart glasses solutions provide surgical site vision to front of eyes regardless of surgeon’s position and orientation.

The COVID-19 pandemic is having the significant positive impact on the surgical smart glasses market. COVID-19 vaccine has been developed by several pharmaceutical companies and is vaccination has been started. Thus, the surgical smart glasses market is growing due to the increasing awareness about remote clinical support as an option for surgeons, which is also anticipated to have a positive impact on the other segments of the market in the coming months. On the other side, disruptions in the supply chain caused due to the halt in global operations are hindering the market growth.

Increasing Number of Surgical Procedures Contribute Significantly to Market Growth

The rising prevalence of chronic diseases such as cancer, diabetes, cardiovascular diseases (CVDs), stroke, and kidney diseases is expected to have a positive impact on the number of surgical procedures across the globe.

As per the WHO, CVDs are the leading cause of mortality, and they result in the deaths of ~17.9 million people every year across the world. Moreover, as per the International Diabetes Federation’s Diabetes Atlas Ninth edition 2019, about 463 million adults (aged 20–79 years) had diabetes in 2019, and the number is expected to reach 700 million by 2045.

Such high prevalence of chronic conditions is expected to drive the number of surgical procedures, which will ultimately boost the growth of the surgical smart glasses market. For instance, according to the data published by the Obesity and Metabolic Surgery Society of India in 2020, the number of bariatric surgeries in India increased by around 86.7% during 2014–2018 in India.

This development is gaining traction as Vuzix hosts its first-ever demonstration in Mandarin – expanding its reach in the Asia-Pacific.

In partnership with SoftFoundry International and Porsche China, Vuzix brings its M-Series line to one of the world’s largest markets.

Why the demonstration is so monumental:

Remote Expertise

Available on the Vuzix M400 and M4000, the FacePro Xpert System ushers in the next-generation of remote support. Businesses can use the technology to train employees, giving instructors access to their student’s point-of-view. Equipped with 4K video, the technology has the potential to significantly reduce corporate expenses. Mass adoption could eliminate “traveling for work.”

Porsche-formance

The premium automaker already benefits from the technology – connecting US-based team members with colleagues in China. This has reduced operating costs for the Vuzix customer, as technical experts can walk trainees through even the most complex procedures. This is performed hands-free from over 7,000 miles away.

Innovative Heartland

China’s steadily-growing economy has an innovative reputation. From electric cars to contactless payment tech, the marketplace of over 1 billion doesn’t shy away from advanced devices that improve the average quality of life. Businesses are expected to embrace Smart Glasses first…but consumers shouldn’t be far behind.

A hands-free world

Supporting 14 different languages, FacePro on Vuzix Smart Glasses is primed to be used across the globe.

The software is making a name for itself in several industries, including healthcare, manufacturing and telecom.

Vuzix is engaged in the design, manufacture, marketing and sale of augmented reality wearable display and computing devices. These display devices are worn like eyeglasses and typically include cameras, sensors, and a computer that enable the user to view, record and interact with video and digital content, allowing their wearers to interact and perform work hands-free. The key growth areas for the Company and its products currently are in the enterprise, medical, consumer electronics, OEM, defense and security markets where we help our customers increase productivity; eliminate travel costs; lower carbon footprint; improves worker safety and protect worker health. With the industry’s broadest product line of smart glasses, Vuzix is now working on it next generation products which include its see-through waveguide optics while implementing the latest micro-LED display technology, as pictured below.

Annual Russell indexes reconstitution captures the 4,000 largest US stocks as of May 7, ranking them by total market capitalization. Membership in the US all-cap Russell 3000® Index, which remains in place for one year, means automatic inclusion in the large-cap Russell 1000® Index or small-cap Russell 2000® Index as well as the appropriate growth and value style indexes. FTSE Russell determines membership for its Russell indexes primarily by objective, market-capitalization rankings and style attributes.

“We are pleased to see our shares added to the Russell Indexes, one of the leading market performance benchmarks in North America,” said Paul Travers, Vuzix Chief Executive Officer. “We view this as an important milestone and one that demonstrates Vuzix’ growing market leadership and enterprise value.  At the same time, we feel this should help raise awareness and the appeal of Vuzix within the institutional investment community.”

Russell indexes are widely used by investment managers and institutional investors for index funds and as benchmarks for active investment strategies. Approximately $10.6 trillion in assets are benchmarked against Russell’s US indexes. Russell indexes are part of FTSE Russell, a leading global index provider.

For more information on the Russell 3000® Index and the Russell indexes reconstitution, go to the “Russell Reconstitution” section on the FTSE Russell website.

FirstNet, built with AT&T, is a public-private partnership with the First Responder Network Authority (FirstNet Authority) – an independent agency within the federal government. It’s designed with and for first responders and the public safety agencies and extended community that could be called on to support them.

The ThirdEye X2 MR Glasses and RespondEye software support access to the physically separate and dedicated FirstNet network core, which enables First Priority® capabilities on FirstNet – always-on priority and, for first responders, preemption – and the FirstNet Band 14 spectrum. FirstNet eligible customers can feel confident the X2 MR Glasses combined with FirstNet services will provide the necessary critical connectivity in a reliable, highly secure and cost-effective manner.

As part of the agreement through the FirstNet Embedded IoT Program – a program that allows for industry leading FirstNet eligible devices to be combined with FirstNet to create a bundled, end-to-end solution for a single monthly fee for first responders, public safety agencies and extended community – the ThirdEye solution will include FirstNet connectivity.

This collaboration comes after ThirdEye successfully launched its pilot programs with the Marcus Hook and Upper Merion Fire Departments in Pennsylvania. ThirdEye’s X2 MR Glasses with the RespondEye software help prevent first responders from coming in contact with potential COVID-19 patients using the attachable FLIR (forward-looking infrared) thermal sensor to detect elevated body temperatures – one of the symptoms of the disease. By using the smart glasses, medics can be hands-free and receive digital information displayed live in their field of view.

“Our telehealth software assists first responders with their everyday tasks while also equipping them with the technology to help protect them from COVID-19,” said Nick Cherukuri, CEO and founder of ThirdEye. “FirstNet provides first responders access to the high-speed and reliable connectivity when facing emergencies. We continue to update the RespondEye’s features upon requests from the EMS teams as we work toward making the implementation of AR in healthcare the new norm.”

By wearing the smart glasses, a paramedic in the field can directly contact a doctor for assistance. The doctor can then stream live video from the scene to be displayed on a computer, tablet or smartphone at the hospital via the RespondEye platform. This allows the doctor to assess a patient and make treatment recommendations without having face-to-face contact. The RespondEye software is cross-compatible and runs on ThirdEye X2 MR Glasses as well as iOS/Android phones, tablets and web browsers. The software is currently free to first responders with tiered pricing for data usage being released soon.

“We’re pleased to welcome ThirdEye as a member of our FirstNet IoT Embedded Program,” said Scott Agnew, assistant vice president, product marketing, FirstNet Program at AT&T. “This allows us to further our mission to deploy, operate, maintain, and enhance the only nationwide wireless platform dedicated to public safety and the extended community that support public safety response.”

At just 300 grams, ThirdEye’s X2 MR Glasses are the lightest mixed reality glasses on the market and are suited for comfortable, extensive wear in all conditions. The glasses run on Android 9.0 and are powered by a Snapdragon xR1 Qualcomm processor chip. ThirdEye also developed custom augmented reality first responder software via its RespondEye software platform suite, which connects to the EMS backend systems. This software combined with ThirdEye’s HIPAA certification enables first responders to access all of their data on demand and safely via the X2 MR Glasses.

ThirdEye achieved HIPAA certification for its X2 MR Glasses and RespondEye Platform, HIPAA compliance is a requirement for vendors that handle any patient medical data to ensure the security of the patient’s information as well as confidentiality.

Ships regularly undergo large-scale retrofits, but shipowners rarely have design data in digital form at hand. To design ship modifications, an engineering accuracy of the as-built ship geometry is required, which means each vessel must be 3D laser-scanned. ShipReality, a company specialised in AR/VR ship design automation and remote ops, synthesises these large ship laser scans with its CAD software to design directly in 3D, resulting in merged models of CAD in the as-built ship geometry point clouds.

“We want to speed up and optimise retrofit designs for 60,000 ships that require greenhouse gas emissions reduction, energy conversions & ballast water treatment system (BWTS) retrofits in the coming years”, said Georgios Bourtzos, CEO and co-founder of ShipReality. “A major challenge we faced designing directly in large point clouds was visualising entire vessels layered with resulting 3D designs for immersive design reviews on mobile XR devices like Oculus Quest 2 and HoloLens 2.”

Exploring the use of point clouds in AR

Point clouds are precise models of real environments based on 3D laser or photogrammetry scanning. Objects and space are represented in the form of “points”. Millions of such points combined formulate a point cloud scan. The scan is then imported into a 3D modeling platform with the purpose of creating an as-built model. Common CAD software used for ship designs, although incorporating 3D laser scans, still rely on 2D projections and screens to visualise and design in 3D. This often results in incompatibilities with the existing ship geometry, which are only realised during installation, creating substantial delays and high additional costs.

Visualisation in 3D is a key issue to address these problems. However, visualising large point clouds requires substantial CPU and graphics power. The performance requirements are simply too high for rendering it locally on a mobile XR device. It would cause an extremely low frame rate and even software crashes. Dealing with large datasets, ShipReality had to find a solution that could surpass the limited memory, CPU and GPU resources of mobile devices.

XR streaming solution that supports point cloud

XR streaming technology outsources the rendering process to a local server or the cloud. But not every solution on the market supports ultra large data or point clouds. “We integrated the ISAR SDK into our solution to stream large 3D laser scans merged with CAD retrofit designs to a HoloLens 2,” said Mr Bourtzos. ISAR (Interactive Streaming for Augmented Reality) is a unique remote rendering software component that allows to stream entire augmented and virtual reality applications in real time. “The simplicity and ease of integration of the software development kit worked seamlessly with our large models.”

Integrating ISAR into their engine and software, ShipReality was able to visualise a massive model containing more than one billion points. Layered on top was also BWTS CAD design data created by the company’s ShipMR-design software with additional five million polygons. To compare, a mobile XR device could only render about one and half million polygons locally. As the remote rendering server ShipReality used a moderate gaming laptop and the local WiFi, broadcasting on 2.4GHz band. More performance and bandwidth enable even greater visualisations.

Next level immersive experiences

“ISAR has amazing potential for AR/VR visualisation of massive digital twins and real-time monitoring of projected complex 3D designs merged with as-built environments in shipping and other industries,” said Mr Bourtzos. ShipReality is now able to:

• visualise ship models that are only suited for high-performance processing
• capture large assets 1:1 and integrate complex 3D CAD designs/data
• visualise detailed models for spatial analytics in augmented reality

“ISAR can save us a lot of time and resources because we can directly use point clouds in mobile AR: some pre-processing steps can be avoided.”

Data availability, accuracy, density, and size of 3D point clouds are also forecast to vastly increase within the next years. “To realise the full potential of immersive point cloud experiences, streaming will play a major role,” concluded Georgios Bourtzos.