High Quality Neural Relighting using Practical Zonal Illumination
No Thumbnail Available
Date
2024
Journal Title
Journal ISSN
Volume Title
Publisher
The Eurographics Association
Abstract
We present a method for high-quality image-based relighting using a practical limited zonal illumination field. Our setup can be implemented with commodity components with no dedicated hardware. We employ a set of desktop monitors to illuminate a subject from a near-hemispherical zone and record One-Light-At-A-Time (OLAT) images from multiple viewpoints. We further extrapolate sampling of incident illumination directions beyond the frontal coverage of the monitors by repeating OLAT captures with the subject rotation in relation to the capture setup. Finally, we train our proposed skip-assisted autoencoder and latent diffusion based generative method to learn a high-quality continuous representation of the reflectance function without requiring explicit alignment of the data captured from various viewpoints. This method enables smooth lighting animation for high-frequency reflectance functions and effectively manages to extend incident lighting beyond the practical capture setup's illumination zone. Compared to state-of-the-art methods, our approach achieves superior image-based relighting results, capturing finer skin pore details and extending to passive performance video relighting.
Description
CCS Concepts: Computing methodologies -> Reflectance modeling; Image-based rendering; Computational photography
@inproceedings{10.2312:sr.20241150,
booktitle = {Eurographics Symposium on Rendering},
editor = {Haines, Eric and Garces, Elena},
title = {{High Quality Neural Relighting using Practical Zonal Illumination}},
author = {Lin, Arvin and Lin, Yiming and Li, Xiaohui and Ghosh, Abhijeet},
year = {2024},
publisher = {The Eurographics Association},
ISSN = {1727-3463},
ISBN = {978-3-03868-262-2},
DOI = {10.2312/sr.20241150}
}