ICG based Augmented-Reality-System for Sentinel Lymph Node Biopsy

Abstract

In this paper we introduce a novel augmented-reality (AR) system for the sentinel lymph node (SLN) biopsy. The AR system consists of a cubic recording device with integrated stereo near-infrared (NIR) and stereo color cameras, an head mounted display (HMD) for visualizing the SLN information directly into the physicians view and a controlling software application. The labeling of the SLN is achieved using the fluorescent dye indocyanine green (ICG). The dye accumulates in the SLN where it is excited to fluorescence by applying infrared light. The fluorescence is recorded from two directions by the NIR stereo cameras using appropriate filters. Applying the known rigid camera geometry, an ICG depth map can be generated from the camera images, thus creating a live 3D representation of the SLN. The representation is then superimposed to the physicians field of view, by applying a series of coordinate system transformations, that are determined in four separate system calibration steps. To compensate for the head motion, the recording systems is continuously tracked by a single camera on the HMD using fiducial markers. Because the system does not require additional monitors, the physicians attention is kept solely on the operation site. This can potentially decrease the intervention time and render the procedure safer for the patient.