Adaptive Grids for Neural Scene Representation
No Thumbnail Available
Date
2024
Journal Title
Journal ISSN
Volume Title
Publisher
The Eurographics Association
Abstract
We introduce a novel versatile approach to enhance the quality of grid-based neural scene representations. Grid-based scene representations model a scene by storing density and color features at discrete 3D points, which offers faster training and rendering than purely implicit methods such as NeRF. However, they require high-resolution grids to achieve high-quality outputs, leading to a significant increase in memory usage. Common standard grids with uniform voxel sizes do not account for the varying complexity of different regions within a scene. This is particularly evident when a highly detailed region or object is present, while the rest of the scene is less significant or simply empty. To address this we introduce a novel approach based on frequency domain transformations for finding the key regions in the scene and then utilize a 2-level hierarchy of grids to allocate more resources to more detailed regions. We also created a more efficient version of this concept, that adapts to a compact grid representation, namely TensoRF, which also works for very few training samples.
Description
CCS Concepts: Computing methodologies → Volumetric models; Rendering; Reconstruction
@inproceedings{10.2312:vmv.20241205,
booktitle = {Vision, Modeling, and Visualization},
editor = {Linsen, Lars and Thies, Justus},
title = {{Adaptive Grids for Neural Scene Representation}},
author = {Pajoum, Barbod and Fox, Gereon and Elgharib, Mohamed and Habermann, Marc and Theobalt, Christian},
year = {2024},
publisher = {The Eurographics Association},
ISBN = {978-3-03868-247-9},
DOI = {10.2312/vmv.20241205}
}