Robust Novel View Synthesis with Color Transform Module

Simple item page
dc.contributor.authorKim, Sang Minen_US
dc.contributor.authorChoi, Changwoonen_US
dc.contributor.authorHeo, Hyeongjunen_US
dc.contributor.authorKim, Young Minen_US
dc.contributor.editorChaine, Raphaëlleen_US
dc.contributor.editorDeng, Zhigangen_US
dc.contributor.editorKim, Min H.en_US
dc.date.accessioned2023-10-09T07:34:01Z
dc.date.available2023-10-09T07:34:01Z
dc.date.issued2023
dc.description.abstractThe advancements of the Neural Radiance Field (NeRF) and its variants have demonstrated remarkable capabilities in generating photo-realistic novel views from a small set of input images. While recent works suggest various techniques and model architectures that enhance speed or reconstruction quality, little attention is paid to exploring the RGB color space of input images. In this paper, we propose a universal color transform module that can maximally harness the captured evidence for the neural networks at hand. The color transform module utilizes an encoder-decoder framework that maps the RGB color space into a new latent space, enhancing the expressiveness of the input domain. We attach the encoder and the decoder at the input and output of a NeRF model of choice, respectively, and jointly optimize them to maintain the cycle consistency of the proposed transform, in addition to minimizing the reconstruction errors in the feature domain. Our comprehensive experiments demonstrate that the learned color space can significantly improve the quality of reconstructions compared to the conventional RGB representation. Its benefits are particularly pronounced in challenging scenarios characterized by low-light environments and scenes with low-textured regions. The proposed color transform pushes the boundaries of limitations in the input domain and offers a promising avenue for advancing the reconstruction capabilities of various neural representations. Source code is available at https://github.com/sangminkim-99/ColorTransformModule.en_US
dc.description.number7
dc.description.sectionheadersNeural Rendering
dc.description.seriesinformationComputer Graphics Forum
dc.description.volume42
dc.identifier.doi10.1111/cgf.14931
dc.identifier.issn1467-8659
dc.identifier.pages14 pages
dc.identifier.urihttps://doi.org/10.1111/cgf.14931
dc.identifier.urihttps://diglib.eg.org:443/handle/10.1111/cgf14931
dc.publisherThe Eurographics Association and John Wiley & Sons Ltd.en_US
dc.subjectCCS Concepts: Computing methodologies -> Reconstruction; Rendering
dc.subjectComputing methodologies
dc.subjectReconstruction
dc.subjectRendering
dc.titleRobust Novel View Synthesis with Color Transform Moduleen_US
Files
Original bundle
Now showing 1 - 1 of 1
No Thumbnail Available
Name:
v42i7_03_14931.pdf
Size:
8.3 MB
Format:
Adobe Portable Document Format
Download
Collections
42-Issue 7