Joint SVBRDF Recovery and Synthesis From a Single Image using an Unsupervised Generative Adversarial Network

dc.contributor.authorZhao, Yezien_US
dc.contributor.authorWang, Beibeien_US
dc.contributor.authorXu, Yanningen_US
dc.contributor.authorZeng, Zhengen_US
dc.contributor.authorWang, Luen_US
dc.contributor.authorHolzschuch, Nicolasen_US
dc.contributor.editorDachsbacher, Carsten and Pharr, Matten_US
dc.date.accessioned2020-06-28T15:27:49Z
dc.date.available2020-06-28T15:27:49Z
dc.date.issued2020
dc.description.abstractWe want to recreate spatially-varying bi-directional reflectance distribution functions (SVBRDFs) from a single image. Pro- ducing these SVBRDFs from single images will allow designers to incorporate many new materials in their virtual scenes, increasing their realism. A single image contains incomplete information about the SVBRDF, making reconstruction difficult. Existing algorithms can produce high-quality SVBRDFs with single or few input photographs using supervised deep learning. The learning step relies on a huge dataset with both input photographs and the ground truth SVBRDF maps. This is a weakness as ground truth maps are not easy to acquire. For practical use, it is also important to produce large SVBRDF maps. Existing algorithms rely on a separate texture synthesis step to generate these large maps, which leads to the loss of consistency be- tween generated SVBRDF maps. In this paper, we address both issues simultaneously. We present an unsupervised generative adversarial neural network that addresses both SVBRDF capture from a single image and synthesis at the same time. From a low-resolution input image, we generate a large resolution SVBRDF, much larger than the input images. We train a generative adversarial network (GAN) to get SVBRDF maps, which have both a large spatial extent and detailed texels. We employ a two-stream generator that divides the training of maps into two groups (normal and roughness as one, diffuse and specular as the other) to better optimize those four maps. In the end, our method is able to generate high-quality large scale SVBRDF maps from a single input photograph with repetitive structures and provides higher quality rendering results with more details compared to the previous works. Each input for our method requires individual training, which costs about 3 hours.en_US
dc.description.sectionheadersBxDFs
dc.description.seriesinformationEurographics Symposium on Rendering - DL-only Track
dc.identifier.doi10.2312/sr.20201136
dc.identifier.isbn978-3-03868-117-5
dc.identifier.issn1727-3463
dc.identifier.pages53-66
dc.identifier.urihttps://doi.org/10.2312/sr.20201136
dc.identifier.urihttps://diglib.eg.org:443/handle/10.2312/sr20201136
dc.publisherThe Eurographics Associationen_US
dc.rightsAttribution 4.0 International License
dc.rights.urihttps://creativecommons.org/licenses/by/4.0/
dc.subjectComputing methodologies
dc.subjectTexturing
dc.titleJoint SVBRDF Recovery and Synthesis From a Single Image using an Unsupervised Generative Adversarial Networken_US
Files
Original bundle
Now showing 1 - 2 of 2
Loading...
Thumbnail Image
Name:
053-066.pdf
Size:
558.25 MB
Format:
Adobe Portable Document Format
Loading...
Thumbnail Image
Name:
paper1023_dlonlytrack_supplementary.pdf
Size:
231.45 MB
Format:
Adobe Portable Document Format