The newest surgical assistants in operating rooms? AR glasses

How AR glasses are improving surgical training and collaboration

A survey conducted by AR glasses manufacturer Vuzix found that many surgeons in the U.S. are eagerly adopting the technology.

And the medical AR glasses market is continuing to grow. Besides Vuzix, products we’re excited about are NuLoupes’ AR magnifying glasses and ARSpectra’s fluorescence image-based surgery AR glasses.

One of the most common applications of AR in surgery is to keep the surgeon’s most important tools—their two hands—free for the task at hand. Within their field of view, they can access in-depth information they may need for their case. No need to check a separate computer screen.

Beyond individual surgeons starting to use the tech, AR’s big impact in surgery is on education. With AR, medical students can remotely observe what surgeon-instructors wearing the glasses are doing.

"As the doctor is doing the operation, students from around the world can log into a HIPAA-compliant livestream and the doctor can describe each step along the way," Vuzix CEO Paul Travers said.

And on the flip side, attendings can closely watch students’ work in the OR to swiftly make adjustments and recommendations.

"The glasses merge two different views into one single view, so I can observe what the trainee is doing," Johns Hopkins professor Ahmed Ghazi said. "They're able to see my hands instructing them on how to maneuver through a certain part of the procedure."

Plus, this capacity for collaboration goes beyond medical education. Now, if a remote expert based on another continent is to consult on a case, they no longer need to book a flight. They can simply log in to the AR headset-wearing surgeon’s live stream—and even draw or annotate over the view in real-time.

"It’s just like he’s standing right next to the doctor in the operating room in South Africa — he can draw on what the doctor is seeing on the overlay of the 4K broadcast-quality camera feed and say, ‘Cut this tissue first,’ or, ‘Use this stent instead of that one,’" Travers said. 

AR and AI in the OR

As it turns out, AR glasses can be even more useful to surgeons—and the broader medical community—when paired with artificial intelligence.

After using AR glasses to record a procedure, some hospitals are using AI to compile and analyze the recorded surgeries. AI may then be able to work off the learnings from all of this data to provide suggestions during real-life operations.

Travers provided an example of how Vuzix’s AI-enabled partnership with virtual surgery platform Proximie can work in practice: “For example, if a surgeon is putting in a stent during heart surgery, the AI engine can provide recommendations for the best type of stent for that particular patient, based on the last thousand similar operations."

With the marriage of AI and AR in surgery, an individual doctor’s surgical experience extends beyond their own lived hours in the OR.

Potential limitations: dropped connections

One of the biggest potential challenges in using AR glasses in surgery is connectivity. 

In an ideal world, strong internet connections across medical centers would prevent lags or dropped calls. However, as the pandemic’s widespread use of video-calling technology showed us, sometimes lags and dropped calls are the price of telecommuting. 

Of course, the risk of a dropped call while performing a high-risk surgery is understandably higher than while doing an investor presentation or during an all-hands call. 

Ghazi acknowledged this risk, saying: "We would have to go through tests to make 100% sure there are no dropped calls and no technical issues, compared to other types of virtual or augmented reality platforms.” 

But sometimes, the weak link is not the technology itself—it’s the internet connection.

In 2020, the Peterson-KFF Health System Tracker published an interesting analysis of how internet connectivity affects healthcare access for patients. The takeaway: a solid internet connection is becoming more and more crucial for healthcare access.

Given the increasing use of AR glasses for clinical care, we’d extend that argument to say that ensuring health centers across the world have reliable, fast internet must be a public health policy priority. As AR-enabled surgery becomes the new gold standard, we must advocate for even the most remote areas to get access to cutting-edge care.