Zhang, XianyaoRöthlin, GerhardZhu, ShilinAydin, Tunç OzanSalehi, FarnoodGross, MarkusPapas, MariosBermano, Amit H.Kalogerakis, Evangelos2024-04-302024-04-3020241467-8659https://doi.org/10.1111/cgf.15050https://diglib.eg.org/handle/10.1111/cgf15050We present a kernel-predicting neural denoising method for path-traced deep-Z images that facilitates their usage in animation and visual effects production. Deep-Z images provide enhanced flexibility during compositing as they contain color, opacity, and other rendered data at multiple depth-resolved bins within each pixel. However, they are subject to noise, and rendering until convergence is prohibitively expensive. The current state of the art in deep-Z denoising yields objectionable artifacts, and current neural denoising methods are incapable of handling the variable number of depth bins in deep-Z images. Our method extends kernel-predicting convolutional neural networks to address the challenges stemming from denoising deep-Z images. We propose a hybrid reconstruction architecture that combines the depth-resolved reconstruction at each bin with the flattened reconstruction at the pixel level. Moreover, we propose depth-aware neighbor indexing of the depth-resolved inputs to the convolution and denoising kernel application operators, which reduces artifacts caused by depth misalignment present in deep-Z images. We evaluate our method on a production-quality deep-Z dataset, demonstrating significant improvements in denoising quality and performance compared to the current state-of-the-art deep-Z denoiser. By addressing the significant challenge of the cost associated with rendering path-traced deep-Z images, we believe that our approach will pave the way for broader adoption of deep-Z workflows in future productions.CCS Concepts: Computing methodologies->Ray tracing; Image processingComputing methodologies>Ray tracingImage processing10.1111/cgf.1505018 pages