A Prototype Holographic Augmented Reality Interface for Image-Guided Prostate Cancer Interventions
Loading...
Date
2018
Journal Title
Journal ISSN
Volume Title
Publisher
The Eurographics Association
Abstract
Motivated by the potential of holographic augmented reality (AR) to offer an immersive 3D appreciation of morphology and anatomy, the purpose of this work is to develop and assess an interface for image-based planning of prostate interventions with a head-mounted display (HMD). The computational system is a data and command pipeline that links a magnetic resonance imaging (MRI) scanner/data and the operator, that includes modules dedicated to image processing and segmentation, structure rendering, trajectory planning and spatial co-registration. The interface was developed with the Unity3D Engine (C#) and deployed and tested on a HoloLens HMD. For ergonomics in the surgical suite, the system was endowed with hands-free interactive manipulation of images and the holographic scene via hand gestures and voice commands. The system was tested in silico using MRI and ultrasound datasets of prostate phantoms. The holographic AR scene rendered by the HoloLens HMD was subjectively found superior to desktop-based volume or 3D rendering with regard to structure detection and appreciation of spatial relationships, planning access paths and manual co-registration of MRI and Ultrasound. By inspecting the virtual trajectory superimposed to rendered structures and MR images, the operator observes collisions of the needle path with vital structures (e.g. urethra) and adjusts accordingly. Holographic AR interfacing with wireless HMD endowed with hands-free gesture and voice control is a promising technology. Studies need to systematically assess the clinical merit of such systems and needed functionalities.
Description
@inproceedings{10.2312:vcbm.20181225,
booktitle = {Eurographics Workshop on Visual Computing for Biology and Medicine},
editor = {Puig Puig, Anna and Schultz, Thomas and Vilanova, Anna and Hotz, Ingrid and Kozlikova, Barbora and Vázquez, Pere-Pau},
title = {{A Prototype Holographic Augmented Reality Interface for Image-Guided Prostate Cancer Interventions}},
author = {Morales Mojica, Cristina M. and Velazco Garcia, Jose D. and Karkoub, Mansour and Leiss, Ernst L. and Seimenis, Ioannis and Tsekos, Nikolaos V. and Navkar, Nikhil V. and Balakrishnan, Shidin and Abinahed, Julien and El Ansari, Walid and Al-Rumaihi, Khalid and Darweesh, Adham and Al-Ansari, Abdulla and Gharib, Mohamed},
year = {2018},
publisher = {The Eurographics Association},
ISSN = {2070-5786},
ISBN = {978-3-03868-056-7},
DOI = {10.2312/vcbm.20181225}
}