SEE4D: Pose-Free 4D Generation via Auto-Regressive Video Inpainting

Simple item page
dc.contributor.authorLu, Dongyue
dc.contributor.authorLiang, Ao
dc.contributor.authorHuang, Tianxin
dc.contributor.authorFu, Xiao
dc.contributor.authorZhao, Yuyang
dc.contributor.authorMa, Baorui
dc.contributor.authorPan, Liang
dc.contributor.authorYin, Wei
dc.contributor.authorKong, Lingdong
dc.contributor.authorOoi, Wei Tsang
dc.contributor.authorLiu, Ziwei
dc.contributor.editorMasia, Belen
dc.contributor.editorThies, Justus
dc.date.accessioned2026-04-17T11:54:16Z
dc.date.available2026-04-17T11:54:16Z
dc.date.issued2026
dc.description.abstractImmersive applications call for synthesizing spatiotemporal 4D content from casual videos without costly 3D supervision. Existing video-to-4D methods typically rely on manually annotated camera poses, which are labor-intensive and brittle for inthe-wild footage. Recent warp-then-inpaint approaches mitigate the need for pose labels by warping input frames along a novel camera trajectory and using an inpainting model to fill missing regions, thereby depicting the 4D scene from diverse viewpoints. However, this trajectory-to-trajectory formulation often entangles camera motion with scene dynamics and complicates both modeling and inference. We introduce SEE4D, a pose-free, trajectory-to-camera framework that replaces explicit trajectory prediction with rendering to a bank of fixed virtual cameras, thereby separating camera control from scene modeling. A viewconditional video inpainting model is trained to learn a robust geometry prior by denoising realistically synthesized warped images and to inpaint occluded or missing regions across virtual viewpoints, eliminating the need for explicit 3D annotations. Building on this inpainting core, we design a spatiotemporal autoregressive inference pipeline that traverses virtual-camera splines and extends videos with overlapping windows, enabling coherent generation at bounded per-step complexity. We validate See4D on cross-view video generation and sparse reconstruction benchmarks. Across quantitative metrics and qualitative assessments, our method achieves superior generalization and improved performance relative to pose- or trajectory-conditioned baselines, advancing practical 4D world modeling from casual videos.
dc.description.number2
dc.description.sectionheadersTemporal Vision: Video Generation, Pose, and Narrative
dc.description.seriesinformationComputer Graphics Forum
dc.description.volume45
dc.identifier.doi10.1111/cgf.70345
dc.identifier.issn1467-8659
dc.identifier.pages13 pages
dc.identifier.urihttps://diglib.eg.org/handle/10.1111/cgf70345
dc.identifier.urihttps://doi.org/10.1111/cgf.70345
dc.publisherThe Eurographics Association and John Wiley & Sons Ltd.
dc.rightsCC-BY-4.0
dc.rights.urihttps://creativecommons.org/licenses/by/4.0/
dc.subjectCCS Concepts: Computing methodologies → Reconstruction; Appearance and texture representations;
dc.subjectCCS Concepts
dc.subjectComputing methodologies → Reconstruction
dc.subjectAppearance and texture representations
dc.titleSEE4D: Pose-Free 4D Generation via Auto-Regressive Video Inpainting
Files
Original bundle
Now showing 1 - 2 of 2
Thumbnail Image
Name:
cgf70345.pdf
Size:
17.05 MB
Format:
Adobe Portable Document Format
Download
Thumbnail Image
Name:
paper1201_mm1.pdf
Size:
33.51 MB
Format:
Adobe Portable Document Format
Download
Collections
45-Issue 2
EG 2026 - Full Papers - CGF 45-Issue 2