CrossComm’s AR and VR potential for surgery consultations

Conventional wisdom holds that technology makes medical care less human, less personal. But watching patient Wanda Touard with virtual reality goggles on her head and controllers in her hands, exploring the inside of her brain for the first time, gives one the distinct impression that nothing could be more personal than this high-tech tool. “Unreal! Amazing!” exclaimed Touard as she virtually traveled along the left side of the bulging aneurysm that required her to have brain surgery in November.

“This is absolutely weird, but absolutely wonderful. My aneurysm looks like a large sweet potato in the middle of my blood vessels. “If this technology had been around six months ago, would I have wanted to see this before my surgery? Absolutely!”

If doctors had heard 10 years ago that one day the very same techies who create video games would be changing the face of surgery, it’s doubtful they would have taken the prediction seriously. But these days, a traditional MRI or CT scan just doesn’t tell the whole story.

Brain aneurysms, which look like bulging balloons along the artery, are due to weak arterial walls and can be exceedingly dangerous. If they rupture, an internal brain bleed can be fatal. That’s why it’s imperative that a neurosurgeon who treats aneurysms has the most optimal view of it before deciding on a course of action. An MRI of patient Wanda Touard’s blood vessels and aneurysm are superimposed on a photo of Buy Now her face at Ochsner Medical Center, a method called ‘augmented reality.’ Touard was treated successfully for the aneurysm using high-tech imaging. Advocate photo by Shawn Fink “This is beyond 3-D printing of the blood vessels and the aneurysm, because with virtual reality, this technology allows me to go inside the blood vessels,” said Dr. Edison Valle, a cerebrovascular neurosurgeon at Ochsner Medical Center. “I can navigate through the patient’s brain just as I would in surgery,” he said.

Doing a “walk-through” ahead of time makes for safer surgery. Seeing the angles of the blood vessels and the depth of the aneurysm helps doctors plan the least invasive and most effective operation, Valle said.

Putting this technology in place at Ochsner was the idea of a neurologist who has a decided affinity for video game technology and the wherewithal to connect tech geeks and medical professionals in an unusual way. “When I moved to New Orleans, I found out that this place is one of the premiere video game development arenas,” said Dr. Korak Sarkar, an Ochsner neurologist. “Twitch is a startup by a founder in Kenner, which televises video games. Amazon just bought this company for slightly under a billion dollars.

“One of the biggest indigenous companies here is called TurboSquid, which is basically an online marketplace for 3-D assets, and one of their co-founders is also on the board of Ochsner,” he said. “Since I go to video game meetups, I acquired some contacts there for people in New Orleans who could do what I wanted them to do and make virtual reality a natural part of the clinical setting.”

Enter Mike Harris, the lead immersive technology developer in New Orleans at CrossComm, a North Carolina based consultancy which creates mobile applications.

It has become his job to turn Sarkar’s dream into a virtual and augmented reality. “If you can imagine an MRI as a stack of individual slices, you can take that stack and via software pull out all of the salient parts,” said Harris. “You now have a collection of data points that can be used to create a 3-D image, but instead of printing them, we’re bringing them into a virtual environment and allowing the physician to manipulate them any way he chooses.” That can mean virtual reality, with goggles, allowing the wearer to “walk” through his or her own anatomy.

Or it can mean augmented reality, in which virtual information is overlaid onto the real world. (Think Pokémon GO, the game everyone played on his phone in which the Pokémon character popped up as part of the real world environment.) “For medical purposes, we can now overlay digital renderings over the patient’s real anatomy,” Harris said. “We now even have augmented reality glasses which project images into your own field of view.”

It’s all a tremendous help in analyzing the individual cases of patients, the doctor said. Dr. James Kalyvas, an Ochsner neurosurgeon specializing in spines, foresees such augmented reality aiding spinal surgery in the near future. By rendering the patient’s spine in 3-D and then superimposing that image on the patient in the correct position on the operating table, surgeons can precisely spot the target area during the operation, Kalyvas said.

Ochsner Hospital is the only facility in the Gulf South using this particular technology, but it’s hoped it will soon become the standard of care. “Has all of this newfangled technology been useful?” asked Valle. “I’m now able to adjust what I thought I could do in a surgery to what I can really do, now that I can see absolutely. It has blown my mind.