The Healthcare Potential of Augmented Reality: Virtual Surgery

One of the leaders in the field is the German company ApoQlar, which is developing a software tool named Virtual Surgery Intelligence (VSI). It uses artificial intelligence to render MRI and CT images in 3D. When a surgeon puts on the augmented/mixed reality headset, the 3D images merge virtually with the patient, giving the surgeon a new level of anatomical detail.  This scan can be controlled with gestures and speech commands.

“It displays medical images in a 3D hologram, which the user can freely manipulate round the physical environment,” says Jakub Wlasny, lead developer at ApoQlar. “Those images are automatically fixed onto the body of the patient. This means doctors can slice through those holograms in order to explore the patient’s internal structures, without having to look at any specific devices or screens.”

Alongside its surgical uses, the tool can be used preoperatively to prepare for upcoming surgeries, and post-operatively to evaluate the surgical progression. ApoQlar is also developing two related tools: VSI Patient Education (for informing patients about their condition) and VSI Education (for training surgeons).

After extensive testing, VSI Surgery has now been introduced to a hospital in Hamburg, where surgeons in the head and neck department are using it as a postoperative and preoperative assistant.

“We have found [the tool] gives us a much deeper and more detailed insight into the anatomical structures,” said Dr Hans von Lücken, senior physician. “But also for our interns the VSI is a big help. This way, they can orientate themselves more quickly and the more experienced doctor can better communicate their surgical approaches and procedures.”

The potential of augmented reality for surgical procedures

Augmented reality technology involves ‘augmenting’ the real-world environment with various pieces of computer-generated input (sound, images or information), by way of a screen or headset. Although it is related to virtual reality (VR), there is one key difference: VR places the user in a completely immersive world, detached from reality, while AR overlays information onto the physical setting.

Although AR is only beginning to make its way into the healthcare space, its potential is significant.

From a medical standpoint, one key advantage is that practitioners don’t need to take their focus off the patient. Currently, surgeons need to switch between various reference points – from the person on the operating table to the medical images and patient data displayed on various screens. With applications such as VSI, they can access all the information they need while the patient remains in their field of vision.

“We want to deliver a more friendly and immersive way of working with medical images, helping doctors navigate around the surgical site and orientate those anatomical structures. This makes it quicker for them to proceed with surgeries,” says Wlasny. “We also want to eliminate as many of the other tools that are now used as we can, and put everything in one simple device.”