Pencil Drawing Video Rendering Using Convolutional Networks
Loading...
Date
2019
Authors
Journal Title
Journal ISSN
Volume Title
Publisher
The Eurographics Association and John Wiley & Sons Ltd.
Abstract
Traditional pencil drawing rendering algorithms when applied to video may suffer from temporal inconsistency and showerdoor effect due to the stochastic noise models employed. This paper attempts to resolve these problems with deep learning. Recently, many research endeavors have demonstrated that feed-forward Convolutional Neural Networks (CNNs) are capable of using a reference image to stylize a whole video sequence while removing the shower-door effect in video style transfer applications. Compared with video style transfer, pencil drawing video is more sensitive to the inconsistency of texture and requires a stronger expression of pencil hatching. Thus, in this paper we develop an approach by combining a latest Line Integral Convolution (LIC) based method, specializing in realistically simulating pencil drawing images, with a new feedforward CNN that can eliminate the shower-door effect successfully. Taking advantage of optical flow, we adopt a feature-maplevel temporal loss function and propose a new framework to avoid the temporal inconsistency between consecutive frames, enhancing the visual impression of pencil strokes and tone. Experimental comparisons with the existing feed-forward CNNs have demonstrated that our method can generate temporally more stable and visually more pleasant pencil drawing video results in a faster manner.
Description
@article{10.1111:cgf.13819,
journal = {Computer Graphics Forum},
title = {{Pencil Drawing Video Rendering Using Convolutional Networks}},
author = {Yan, Dingkun and Sheng, Yun and Mao, Xiaoyang},
year = {2019},
publisher = {The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {10.1111/cgf.13819}
}