Deep Shape and SVBRDF Estimation using Smartphone Multi-lens Imaging
No Thumbnail Available
Date
2023
Authors
Journal Title
Journal ISSN
Volume Title
Publisher
The Eurographics Association and John Wiley & Sons Ltd.
Abstract
We present a deep neural network-based method that acquires high-quality shape and spatially varying reflectance of 3D objects using smartphone multi-lens imaging. Our method acquires two images simultaneously using a zoom lens and a wide angle lens of a smartphone under either natural illumination or phone flash conditions, effectively functioning like a single-shot method. Unlike traditional multi-view stereo methods which require sufficient differences in viewpoint and only estimate depth at a certain coarse scale, our method estimates fine-scale depth by utilising an optical-flow field extracted from subtle baseline and perspective due to different optics in the two images captured simultaneously. We further guide the SVBRDF estimation using the estimated depth, resulting in superior results compared to existing single-shot methods.
Description
CCS Concepts: Computing methodologies -> Computational photography; Shape inference; Reflectance modeling
@article{10.1111:cgf.14972,
journal = {Computer Graphics Forum},
title = {{Deep Shape and SVBRDF Estimation using Smartphone Multi-lens Imaging}},
author = {Fan, Chongrui and Lin, Yiming and Ghosh, Abhijeet},
year = {2023},
publisher = {The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {10.1111/cgf.14972}
}