Towards Integrating Multi-Spectral Imaging with Gaussian Splatting

Simple item page
dc.contributor.authorGrün, Josefen_US
dc.contributor.authorMeyer, Lukasen_US
dc.contributor.authorWeiherer, Maximilianen_US
dc.contributor.authorEgger, Bernharden_US
dc.contributor.authorStamminger, Marcen_US
dc.contributor.authorFranke, Linusen_US
dc.contributor.editorEgger, Bernharden_US
dc.contributor.editorGünther, Tobiasen_US
dc.date.accessioned2025-09-24T10:38:00Z
dc.date.available2025-09-24T10:38:00Z
dc.date.issued2025
dc.description.abstractWe present a study of how to integrate color (RGB) and multi-spectral imagery (red, green, red-edge, and near-infrared) into the 3D Gaussian Splatting (3DGS) framework - a state-of-the-art explicit radiance-field-based method for fast and high-fidelity 3D reconstruction from multi-view images [KKLD23]. While 3DGS excels on RGB data, naïve per-band optimization of additional spectra yields poor reconstructions due to inconsistently appearing geometry in the spectral domain. This problem is prominent, even though the actual geometry is the same, regardless of spectral modality. To investigate this, we evaluate three strategies: 1) Separate per-band reconstruction with no shared structure; 2) Splitting optimization, in which we first optimize RGB geometry, copy it, and then fit each new band to the model by optimizing both geometry and band representation; and 3) Joint, in which the modalities are jointly optimized, optionally with an initial RGB-only phase. We showcase through quantitative metrics and qualitative novel-view renderings on multi-spectral datasets the effectiveness of our dedicated optimized Joint strategy, increasing overall spectral reconstruction as well as enhancing RGB results through spectral cross-talk. We therefore suggest integrating multi-spectral data directly into the spherical harmonics color components to compactly model each Gaussian's multi-spectral reflectance. Moreover, our analysis reveals several key trade-offs in when and how to introduce spectral bands during optimization, offering practical insights for robust multi-modal 3DGS reconstruction. The project page and code is located at: meyerls.github.io/towards_multi_spec_splaten_US
dc.description.sectionheadersImaging and Image Processing
dc.description.seriesinformationVision, Modeling, and Visualization
dc.identifier.doi10.2312/vmv.20251237
dc.identifier.isbn978-3-03868-294-3
dc.identifier.pages9 pages
dc.identifier.urihttps://doi.org/10.2312/vmv.20251237
dc.identifier.urihttps://diglib.eg.org/handle/10.2312/vmv20251237
dc.publisherThe Eurographics Associationen_US
dc.rightsAttribution 4.0 International License
dc.rights.urihttps://creativecommons.org/licenses/by/4.0/
dc.subjectCCS Concepts: Computing methodologies → Rendering; Reconstruction; Hyperspectral imaging
dc.subjectComputing methodologies → Rendering
dc.subjectReconstruction
dc.subjectHyperspectral imaging
dc.titleTowards Integrating Multi-Spectral Imaging with Gaussian Splattingen_US
Files
Original bundle
Now showing 1 - 3 of 3
Thumbnail Image
Name:
vmv20251237.pdf
Size:
49.44 MB
Format:
Adobe Portable Document Format
Download
No Thumbnail Available
Name:
paper1024_1.pdf
Size:
225.12 MB
Format:
Adobe Portable Document Format
Download
No Thumbnail Available
Name:
paper1024_2.mov
Size:
560.17 MB
Format:
Video Quicktime
Download
Collections
VMV2025