Application of 3D Gaussian Splatting for Cinematic Anatomy on Consumer Class Devices
No Thumbnail Available
Date
2024
Journal Title
Journal ISSN
Volume Title
Publisher
The Eurographics Association
Abstract
Interactive photorealistic rendering of 3D anatomy is used in medical education to explain the structure of the human body. It is currently restricted to frontal teaching scenarios, where even with a powerful GPU and high-speed access to a large storage device where the data set is hosted, interactive demonstrations can hardly be achieved. We present the use of novel view synthesis via compressed 3D Gaussian Splatting (3DGS) to overcome this restriction, and to even enable students to perform cinematic anatomy on lightweight and mobile devices. Our proposed pipeline first finds a set of camera poses that captures all potentially seen structures in the data. High-quality images are then generated with path tracing and converted into a compact 3DGS representation, consuming < 70 MB even for data sets of multiple GBs. This allows for real-time photorealistic novel view synthesis that recovers structures up to the voxel resolution and is almost indistinguishable from the path-traced images.
Description
CCS Concepts: Computing methodologies->Computer graphics; Rendering
@inproceedings{10.2312:vmv.20241195,
booktitle = {Vision, Modeling, and Visualization},
editor = {Linsen, Lars and Thies, Justus},
title = {{Application of 3D Gaussian Splatting for Cinematic Anatomy on Consumer Class Devices}},
author = {Niedermayr, Simon and Neuhauser, Christoph and Petkov, Kaloian and Engel, Klaus and Westermann, Rüdiger},
year = {2024},
publisher = {The Eurographics Association},
ISBN = {978-3-03868-247-9},
DOI = {10.2312/vmv.20241195}
}