Generative Adversarial Shaders for Real-Time Realism Enhancement

Salmi, Arturo; Cséfalvay, Szabolcs; Imber, James

Generative Adversarial Shaders for Real-Time Realism Enhancement

Files

v42i8_05_14870.pdf (80.86 MB)

mm1032.mp4 (574.85 MB)

Date

2023

Authors

Salmi, Arturo
Cséfalvay, Szabolcs
Imber, James

Publisher

The Eurographics Association and John Wiley & Sons Ltd.

Abstract

Application of realism enhancement methods, particularly in real-time and resource-constrained settings, has been frustrated by the expense of existing methods. These achieve high quality results only at the cost of long runtimes and high bandwidth, memory, and power requirements. We present an efficient alternative: a high-performance, generative shader-based approach that adapts machine learning techniques to real-time applications, even in resource-constrained settings such as embedded and mobile GPUs. The proposed learnable shader pipeline comprises differentiable functions that can be trained in an end-toend manner using an adversarial objective, allowing for faithful reproduction of the appearance of a target image set without manual tuning. The shader pipeline is optimized for highly efficient execution on the target device, providing temporally stable, faster-than-real time results with quality competitive with many neural network-based methods.

CCS Concepts: Computing methodologies -> Computer graphics; Machine learning; Image processing; Rendering; Graphics processors

        @article{10.1111:cgf.14870
,
journal = {Computer Graphics Forum},
title = {{Generative Adversarial Shaders for Real-Time Realism Enhancement
}},
author = {Salmi, Arturo and 
Cséfalvay, Szabolcs and 
Imber, James
},
year = {2023
},
publisher = {The Eurographics Association and John Wiley & Sons Ltd.
},
ISSN = {1467-8659
},
DOI = {10.1111/cgf.14870
}
}