StylePortraitVideo: Editing Portrait Videos with Expression Optimization
Loading...
Date
2022
Journal Title
Journal ISSN
Volume Title
Publisher
The Eurographics Association and John Wiley & Sons Ltd.
Abstract
High-quality portrait image editing has been made easier by recent advances in GANs (e.g., StyleGAN) and GAN inversion methods that project images onto a pre-trained GAN's latent space. However, extending the existing image editing methods, it is hard to edit videos to produce temporally coherent and natural-looking videos. We find challenges in reproducing diverse video frames and preserving the natural motion after editing. In this work, we propose solutions for these challenges. First, we propose a video adaptation method that enables the generator to reconstruct the original input identity, unusual poses, and expressions in the video. Second, we propose an expression dynamics optimization that tweaks the latent codes to maintain the meaningful motion in the original video. Based on these methods, we build a StyleGAN-based high-quality portrait video editing system that can edit videos in the wild in a temporally coherent way at up to 4K resolution.
Description
CCS Concepts: Computing methodologies --> Computer vision; Image manipulation
@article{10.1111:cgf.14666,
journal = {Computer Graphics Forum},
title = {{StylePortraitVideo: Editing Portrait Videos with Expression Optimization}},
author = {Seo, Kwanggyoon and Oh, Seoung Wug and Lu, Jingwan and Lee, Joon-Young and Kim, Seonghyeon and Noh, Junyong},
year = {2022},
publisher = {The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {10.1111/cgf.14666}
}