Neurosymbolic Models for Computer Graphics
No Thumbnail Available
Date
2023
Journal Title
Journal ISSN
Volume Title
Publisher
The Eurographics Association and John Wiley & Sons Ltd.
Abstract
Procedural models (i.e. symbolic programs that output visual data) are a historically-popular method for representing graphics content: vegetation, buildings, textures, etc. They offer many advantages: interpretable design parameters, stochastic variations, high-quality outputs, compact representation, and more. But they also have some limitations, such as the difficulty of authoring a procedural model from scratch. More recently, AI-based methods, and especially neural networks, have become popular for creating graphic content. These techniques allow users to directly specify desired properties of the artifact they want to create (via examples, constraints, or objectives), while a search, optimization, or learning algorithm takes care of the details. However, this ease of use comes at a cost, as it's often hard to interpret or manipulate these representations. In this state-of-the-art report, we summarize research on neurosymbolic models in computer graphics: methods that combine the strengths of both AI and symbolic programs to represent, generate, and manipulate visual data. We survey recent work applying these techniques to represent 2D shapes, 3D shapes, and materials & textures. Along the way, we situate each prior work in a unified design space for neurosymbolic models, which helps reveal underexplored areas and opportunities for future research.
Description
CCS Concepts: Computing methodologies -> Shape modeling; Reflectance modeling; Texturing; Neural networks; Computer vision; Software and its engineering -> Domain specific languages; Programming by example
@article{10.1111:cgf.14775,
journal = {Computer Graphics Forum},
title = {{Neurosymbolic Models for Computer Graphics}},
author = {Ritchie, Daniel and Guerrero, Paul and Jones, R. Kenny and Mitra, Niloy J. and Schulz, Adriana and Willis, Karl D. D. and Wu, Jiajun},
year = {2023},
publisher = {The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {10.1111/cgf.14775}
}