Relighting Humans in the Wild: Monocular Full-Body Human Relighting with Domain Adaptation

dc.contributor.authorTajima, Daichien_US
dc.contributor.authorKanamori, Yoshihiroen_US
dc.contributor.authorEndo, Yukien_US
dc.contributor.editorZhang, Fang-Lue and Eisemann, Elmar and Singh, Karanen_US
dc.date.accessioned2021-10-14T11:12:04Z
dc.date.available2021-10-14T11:12:04Z
dc.date.issued2021
dc.description.abstractThe modern supervised approaches for human image relighting rely on training data generated from 3D human models. However, such datasets are often small (e.g., Light Stage data with a small number of individuals) or limited to diffuse materials (e.g., commercial 3D scanned human models). Thus, the human relighting techniques suffer from the poor generalization capability and synthetic-to-real domain gap. In this paper, we propose a two-stage method for single-image human relighting with domain adaptation. In the first stage, we train a neural network for diffuse-only relighting. In the second stage, we train another network for enhancing non-diffuse reflection by learning residuals between real photos and images reconstructed by the diffuse-only network. Thanks to the second stage, we can achieve higher generalization capability against various cloth textures, while reducing the domain gap. Furthermore, to handle input videos, we integrate illumination-aware deep video prior to greatly reduce flickering artifacts even with challenging settings under dynamic illuminations.en_US
dc.description.number7
dc.description.sectionheadersImage Synthesis and Enhancement
dc.description.seriesinformationComputer Graphics Forum
dc.description.volume40
dc.identifier.doi10.1111/cgf.14414
dc.identifier.issn1467-8659
dc.identifier.pages205-216
dc.identifier.urihttps://doi.org/10.1111/cgf.14414
dc.identifier.urihttps://diglib.eg.org:443/handle/10.1111/cgf14414
dc.publisherThe Eurographics Association and John Wiley & Sons Ltd.en_US
dc.subjectComputing methodologies
dc.subjectImage manipulation
dc.subjectNeural networks
dc.titleRelighting Humans in the Wild: Monocular Full-Body Human Relighting with Domain Adaptationen_US
Files
Original bundle
Now showing 1 - 2 of 2
Loading...
Thumbnail Image
Name:
v40i7pp205-216.pdf
Size:
6.61 MB
Format:
Adobe Portable Document Format
No Thumbnail Available
Name:
paper1028_mm.mp4
Size:
138.86 MB
Format:
Unknown data format
Collections