A Diffusion Approach to Radiance Field Relighting using Multi-Illumination Synthesis
| dc.contributor.author | Poirier-Ginter, Yohan | en_US |
| dc.contributor.author | Gauthier, Alban | en_US |
| dc.contributor.author | Philip, Julien | en_US |
| dc.contributor.author | Lalonde, Jean-François | en_US |
| dc.contributor.author | Drettakis, George | en_US |
| dc.contributor.editor | Garces, Elena | en_US |
| dc.contributor.editor | Haines, Eric | en_US |
| dc.date.accessioned | 2024-06-25T10:19:34Z | |
| dc.date.available | 2024-06-25T10:19:34Z | |
| dc.date.issued | 2024 | |
| dc.description.abstract | Relighting radiance fields is severely underconstrained for multi-view data, which is most often captured under a single illumination condition; It is especially hard for full scenes containing multiple objects. We introduce a method to create relightable radiance fields using such single-illumination data by exploiting priors extracted from 2D image diffusion models. We first fine-tune a 2D diffusion model on a multi-illumination dataset conditioned by light direction, allowing us to augment a single-illumination capture into a realistic - but possibly inconsistent - multi-illumination dataset from directly defined light directions. We use this augmented data to create a relightable radiance field represented by 3D Gaussian splats. To allow direct control of light direction for low-frequency lighting, we represent appearance with a multi-layer perceptron parameterized on light direction. To enforce multi-view consistency and overcome inaccuracies we optimize a per-image auxiliary feature vector. We show results on synthetic and real multi-view data under single illumination, demonstrating that our method successfully exploits 2D diffusion model priors to allow realistic 3D relighting for complete scenes. | en_US |
| dc.description.number | 4 | |
| dc.description.sectionheaders | Relighting | |
| dc.description.seriesinformation | Computer Graphics Forum | |
| dc.description.volume | 43 | |
| dc.identifier.doi | 10.1111/cgf.15147 | |
| dc.identifier.issn | 1467-8659 | |
| dc.identifier.pages | 14 pages | |
| dc.identifier.uri | https://doi.org/10.1111/cgf.15147 | |
| dc.identifier.uri | https://diglib.eg.org/handle/10.1111/cgf15147 | |
| dc.publisher | The Eurographics Association and John Wiley & Sons Ltd. | en_US |
| dc.subject | Keywords: NeRF, Radiance Field, Relighting | |
| dc.subject | NeRF | |
| dc.subject | Radiance Field | |
| dc.subject | Relighting | |
| dc.title | A Diffusion Approach to Radiance Field Relighting using Multi-Illumination Synthesis | en_US |