Real-time Deep Radiance Reconstruction from Imperfect Caches

Huang, Tao; Song, Yadong; Guo, Jie; Tao, Chengzhi; Zong, Zijing; Fu, Xihao; Li, Hongshan; Guo, Yanwen

Real-time Deep Radiance Reconstruction from Imperfect Caches

Files

v41i7pp267-278.pdf (101 MB)

Date

2022

Authors

Huang, Tao
Song, Yadong
Guo, Jie
Tao, Chengzhi
Zong, Zijing
Fu, Xihao
Li, Hongshan
Guo, Yanwen

Publisher

The Eurographics Association and John Wiley & Sons Ltd.

Abstract

Real-time global illumination is a highly desirable yet challenging task in computer graphics. Existing works well solving this problem are mostly based on some kind of precomputed data (caches), while the final results depend significantly on the quality of the caches. In this paper, we propose a learning-based pipeline that can reproduce a wide range of complex light transport phenomena, including high-frequency glossy interreflection, at any viewpoint in real time (> 90 frames per-second), using information from imperfect caches stored at the barycentre of every triangle in a 3D scene. These caches are generated at a precomputation stage by a physically-based offline renderer at a low sampling rate (e.g., 32 samples per-pixel) and a low image resolution (e.g., 64×16). At runtime, a deep radiance reconstruction method based on a dedicated neural network is then involved to reconstruct a high-quality radiance map of full global illumination at any viewpoint from these imperfect caches, without introducing noise and aliasing artifacts. To further improve the reconstruction accuracy, a new feature fusion strategy is designed in the network to better exploit useful contents from cheap G-buffers generated at runtime. The proposed framework ensures high-quality rendering of images for moderate-sized scenes with full global illumination effects, at the cost of reasonable precomputation time. We demonstrate the effectiveness and efficiency of the proposed pipeline by comparing it with alternative strategies, including real-time path tracing and precomputed radiance transfer.

CCS Concepts: Computing methodologies → Ray tracing; Neural networks

        @article{10.1111:cgf.14675
,
journal = {Computer Graphics Forum},
title = {{Real-time Deep Radiance Reconstruction from Imperfect Caches
}},
author = {Huang, Tao and 
Song, Yadong and 
Guo, Jie and 
Tao, Chengzhi and 
Zong, Zijing and 
Fu, Xihao and 
Li, Hongshan and 
Guo, Yanwen
},
year = {2022
},
publisher = {The Eurographics Association and John Wiley & Sons Ltd.
},
ISSN = {1467-8659
},
DOI = {10.1111/cgf.14675
}
}