Semantics-guided Generative Diffusion Model with a 3DMM Model Condition for Face Swapping

Simple item page
dc.contributor.authorLiu, Xiyaoen_US
dc.contributor.authorLiu, Yangen_US
dc.contributor.authorZheng, Yuhaoen_US
dc.contributor.authorYang, Tingen_US
dc.contributor.authorZhang, Jianen_US
dc.contributor.authorWang, Victoriaen_US
dc.contributor.authorFang, Huien_US
dc.contributor.editorChaine, Raphaëlleen_US
dc.contributor.editorDeng, Zhigangen_US
dc.contributor.editorKim, Min H.en_US
dc.date.accessioned2023-10-09T07:34:48Z
dc.date.available2023-10-09T07:34:48Z
dc.date.issued2023
dc.description.abstractFace swapping is a technique that replaces a face in a target media with another face of a different identity from a source face image. Currently, research on the effective utilisation of prior knowledge and semantic guidance for photo-realistic face swapping remains limited, despite the impressive synthesis quality achieved by recent generative models. In this paper, we propose a novel conditional Denoising Diffusion Probabilistic Model (DDPM) enforced by a two-level face prior guidance. Specifically, it includes (i) an image-level condition generated by a 3D Morphable Model (3DMM), and (ii) a high-semantic level guidance driven by information extracted from several pre-trained attribute classifiers, for high-quality face image synthesis. Although swapped face image from 3DMM does not achieve photo-realistic quality on its own, it provides a strong image-level prior, in parallel with high-level face semantics, to guide the DDPM for high fidelity image generation. The experimental results demonstrate that our method outperforms state-of-the-art face swapping methods on benchmark datasets in terms of its synthesis quality, and capability to preserve the target face attributes and swap the source face identity.en_US
dc.description.number7
dc.description.sectionheadersVirtual Humans
dc.description.seriesinformationComputer Graphics Forum
dc.description.volume42
dc.identifier.doi10.1111/cgf.14949
dc.identifier.issn1467-8659
dc.identifier.pages12 pages
dc.identifier.urihttps://doi.org/10.1111/cgf.14949
dc.identifier.urihttps://diglib.eg.org:443/handle/10.1111/cgf14949
dc.publisherThe Eurographics Association and John Wiley & Sons Ltd.en_US
dc.subjectCCS Concepts: Computing methodologies -> Computer graphics; Image manipulation; Computational photography
dc.subjectComputing methodologies
dc.subjectComputer graphics
dc.subjectImage manipulation
dc.subjectComputational photography
dc.titleSemantics-guided Generative Diffusion Model with a 3DMM Model Condition for Face Swappingen_US
Files
Original bundle
Now showing 1 - 2 of 2
No Thumbnail Available
Name:
v42i7_21_14949.pdf
Size:
6.45 MB
Format:
Adobe Portable Document Format
Download
No Thumbnail Available
Name:
paper1106_mm.pdf
Size:
34.78 MB
Format:
Adobe Portable Document Format
Download
Collections
42-Issue 7