Estimating the Pose of a Medical Manikin for Haptic Augmentation of a Virtual Patient in Mixed Reality Training

Scherfgen, David; Schild, Jonas

Estimating the Pose of a Medical Manikin for Haptic Augmentation of a Virtual Patient in Mixed Reality Training

Files

003-004.pdf (7.33 MB)

Date

2021

Authors

Scherfgen, David
Schild, Jonas

Publisher

The Eurographics Association

Abstract

Virtual medical emergency training provides complex while safe interactions with virtual patients. Haptically integrating a medical manikin into virtual training has the potential to improve the interaction with a virtual patient and the training experience. We present a system that estimates the 3D pose of a medical manikin in order to haptically augment a human model in a virtual reality training environment, allowing users to physically touch a virtual patient. The system uses an existing convolutional neural network-based (CNN) body keypoint detector to locate relevant 2D keypoints of the manikin in the images of the stereo camera built into a head-mounted display. The manikin's position, orientation and joint angles are found by non-linear optimization. A preliminary analysis reports an error of 4.3 cm. The system is not yet capable of real-time processing.

        @inproceedings{10.2312:egve.20211334
,
booktitle = {ICAT-EGVE 2021 - International Conference on Artificial Reality and Telexistence and Eurographics Symposium on Virtual Environments - Posters and Demos
},
editor = {Maiero, Jens and Weier, Martin and Zielasko, Daniel
},
title = {{Estimating the Pose of a Medical Manikin for Haptic Augmentation of a Virtual Patient in Mixed Reality Training
}},
author = {Scherfgen, David and 
Schild, Jonas
},
year = {2021
},
publisher = {The Eurographics Association
},
ISSN = {1727-530X
},
ISBN = {978-3-03868-159-5
},
DOI = {10.2312/egve.20211334
}
}