Facial Animation with Disentangled Identity and Motion using Transformers

Simple item page
dc.contributor.authorChandran, Prashanthen_US
dc.contributor.authorZoss, Gasparden_US
dc.contributor.authorGross, Markusen_US
dc.contributor.authorGotardo, Pauloen_US
dc.contributor.authorBradley, Dereken_US
dc.contributor.editorDominik L. Michelsen_US
dc.contributor.editorSoeren Pirken_US
dc.date.accessioned2022-08-10T15:19:57Z
dc.date.available2022-08-10T15:19:57Z
dc.date.issued2022
dc.description.abstractWe propose a 3D+time framework for modeling dynamic sequences of 3D facial shapes, representing realistic non-rigid motion during a performance. Our work extends neural 3D morphable models by learning a motion manifold using a transformer architecture. More specifically, we derive a novel transformer-based autoencoder that can model and synthesize 3D geometry sequences of arbitrary length. This transformer naturally determines frame-to-frame correlations required to represent the motion manifold, via the internal self-attention mechanism. Furthermore, our method disentangles the constant facial identity from the time-varying facial expressions in a performance, using two separate codes to represent neutral identity and the performance itself within separate latent subspaces. Thus, the model represents identity-agnostic performances that can be paired with an arbitrary new identity code and fed through our new identity-modulated performance decoder; the result is a sequence of 3D meshes for the performance with the desired identity and temporal length. We demonstrate how our disentangled motion model has natural applications in performance synthesis, performance retargeting, key-frame interpolation and completion of missing data, performance denoising and retiming, and other potential applications that include full 3D body modeling.en_US
dc.description.number8
dc.description.sectionheadersCapture, Tracking, and Facial Animation
dc.description.seriesinformationComputer Graphics Forum
dc.description.volume41
dc.identifier.doi10.1111/cgf.14641
dc.identifier.issn1467-8659
dc.identifier.pages267-277
dc.identifier.pages11 pages
dc.identifier.urihttps://doi.org/10.1111/cgf.14641
dc.identifier.urihttps://diglib.eg.org:443/handle/10.1111/cgf14641
dc.publisherThe Eurographics Association and John Wiley & Sons Ltd.en_US
dc.subjectCCS Concepts: Computing methodologies --> Motion processing; Shape modeling; Mesh geometry models
dc.subjectComputing methodologies
dc.subjectMotion processing
dc.subjectShape modeling
dc.subjectMesh geometry models
dc.titleFacial Animation with Disentangled Identity and Motion using Transformersen_US
Files
Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
v41i8pp267-277.pdf
Size:
109.63 MB
Format:
Adobe Portable Document Format
Download
Collections
41-Issue 8