Temporally Stable Real-Time Joint Neural Denoising and Supersampling
No Thumbnail Available
Date
2022
Journal Title
Journal ISSN
Volume Title
Publisher
ACM Association for Computing Machinery
Abstract
Recent advances in ray tracing hardware bring real-time path tracing into reach, and ray traced soft shadows, glossy reflections, and diffuse global illumination are now common features in games. Nonetheless, ray budgets are still limited. This results in undersampling, which manifests as aliasing and noise. Prior work addresses these issues separately. While temporal supersampling methods based on neural networks have gained a wide use in modern games due to their better robustness, neural denoising remains challenging because of its higher computational cost. We introduce a novel neural network architecture for real-time rendering that combines supersampling and denoising, thus lowering the cost compared to two separate networks. This is achieved by sharing a single low-precision feature extractor with multiple higher-precision filter stages. To reduce cost further, our network takes low-resolution inputs and reconstructs a high-resolution denoised supersampled output. Our technique produces temporally stable high-fidelity results that significantly outperform state-of-the-art real-time statistical or analytical denoisers combined with TAA or neural upsampling to the target resolution. We introduce a novel neural network architecture for real-time rendering that combines supersampling and denoising, thus lowering the cost compared to two separate networks. This is achieved by sharing a single low-precision feature extractor with multiple higher-precision filter stages. To reduce cost further, our network takes low-resolution inputs and reconstructs a high-resolution denoised supersampled output. Our technique produces temporally stable high-fidelity results that significantly outperform state-of-the-art real-time statistical or analytical denoisers combined with TAA or neural upsampling to the target resolution.
Description
CCS Concepts: Computer systems organization -> Neural Network; Rendering Additional Key Words and Phrases: Kernel prediction, ray tracing, denoising, antialiasing, supersampling, super-resolution, real-time rendering, deep learning
@inproceedings{10.1145:3543870,
booktitle = {Proceedings of the ACM on Computer Graphics and Interactive Techniques},
editor = {Josef Spjut and Marc Stamminger and Victor Zordan},
title = {{Temporally Stable Real-Time Joint Neural Denoising and Supersampling}},
author = {Thomas, Manu Mathew and Liktor, Gabor and Peters, Christoph and Kim, Sungye and Vaidyanathan, Karthik and Forbes, Angus G.},
year = {2022},
publisher = {ACM Association for Computing Machinery},
ISSN = {2577-6193},
ISBN = {},
DOI = {10.1145/3543870}
}