Adaptive Grids for Neural Scene Representation

No Thumbnail Available
Date
2024
Journal Title
Journal ISSN
Volume Title
Publisher
The Eurographics Association
Abstract
We introduce a novel versatile approach to enhance the quality of grid-based neural scene representations. Grid-based scene representations model a scene by storing density and color features at discrete 3D points, which offers faster training and rendering than purely implicit methods such as NeRF. However, they require high-resolution grids to achieve high-quality outputs, leading to a significant increase in memory usage. Common standard grids with uniform voxel sizes do not account for the varying complexity of different regions within a scene. This is particularly evident when a highly detailed region or object is present, while the rest of the scene is less significant or simply empty. To address this we introduce a novel approach based on frequency domain transformations for finding the key regions in the scene and then utilize a 2-level hierarchy of grids to allocate more resources to more detailed regions. We also created a more efficient version of this concept, that adapts to a compact grid representation, namely TensoRF, which also works for very few training samples.
Description

CCS Concepts: Computing methodologies → Volumetric models; Rendering; Reconstruction

        
@inproceedings{
10.2312:vmv.20241205
, booktitle = {
Vision, Modeling, and Visualization
}, editor = {
Linsen, Lars
and
Thies, Justus
}, title = {{
Adaptive Grids for Neural Scene Representation
}}, author = {
Pajoum, Barbod
and
Fox, Gereon
and
Elgharib, Mohamed
and
Habermann, Marc
and
Theobalt, Christian
}, year = {
2024
}, publisher = {
The Eurographics Association
}, ISBN = {
978-3-03868-247-9
}, DOI = {
10.2312/vmv.20241205
} }
Citation
Collections