Brush Stroke Synthesis with a Generative Adversarial Network Driven by Physically Based Simulation

Wu, Rundong; Chen, Zhili; Wang, Zhaowen; Yang, Jimei; Marschner, Steve

Brush Stroke Synthesis with a Generative Adversarial Network Driven by Physically Based Simulation

Date

2018

Authors

Wu, Rundong
Chen, Zhili
Wang, Zhaowen
Yang, Jimei
Marschner, Steve

Publisher

ACM

Abstract

We introduce a novel approach that uses a generative adversarial network (GAN) to synthesize realistic oil painting brush strokes, where the network is trained with data generated by a high-fidelity simulator. Among approaches to digitally synthesizing natural media painting strokes, methods using physically based simulation by far produce the most realistic visual results and allow the most intuitive control of stroke variations. However, accurate physics simulations are known to be computationally expensive and often cannot meet the performance requirements of painting applications. A few existing simulation-based methods have managed to reach real-time performance at the cost of lower visual quality resulting from simplified models or lower resolution. In our work, we propose to replace the expensive fluid simulation with a neural network generator. The network takes the existing canvas and new brush trajectory information as input and produces the height and color of the paint surface as output. We build a large painting sample training dataset by feeding random strokes from artists' recordings into a high quality offline simulator. The network is able to produce visual quality comparable to the offline simulator with better performance than the existing real-time oil painting simulator. Finally, we implement a real-time painting system using the trained network with stroke splitting and patch blending and show artworks created with the system by artists. Our neural network approach opens up new opportunities for real-time applications of sophisticated and expensive physically based simulation.

        @inproceedings{10.1145:3229147.3229150
,
booktitle = {Expressive: Computational Aesthetics, Sketch-Based Interfaces and Modeling, Non-Photorealistic Animation and Rendering
},
editor = {Aydın, Tunç and Sýkora, Daniel
},
title = {{Brush Stroke Synthesis with a Generative Adversarial Network Driven by Physically Based Simulation
}},
author = {Wu, Rundong and 
Chen, Zhili and 
Wang, Zhaowen and 
Yang, Jimei and 
Marschner, Steve
},
year = {2018
},
publisher = {ACM
},
ISSN = {2079-8679
},
ISBN = {978-1-4503-5892-7
},
DOI = {10.1145/3229147.3229150
}
}

URI

https://doi.org/10.1145/3229147.3229150
https://diglib.eg.org:443/handle/10.1145/3229147-3229150

Collections

Expressive 2018

Full item page