Immersive Free‐Viewpoint Panorama Rendering from Omnidirectional Stereo Video
No Thumbnail Available
Date
2023
Authors
Mühlhausen, Moritz
Kappel, Moritz
Kassubeck, Marc
Wöhler, Leslie
Grogorick, Steve
Castillo, Susana
Eisemann, Martin
Magnor, Marcus
Kappel, Moritz
Kassubeck, Marc
Wöhler, Leslie
Grogorick, Steve
Castillo, Susana
Eisemann, Martin
Magnor, Marcus
Journal Title
Journal ISSN
Volume Title
Publisher
© 2023 Eurographics ‐ The European Association for Computer Graphics and John Wiley & Sons Ltd.
Abstract
In this paper, we tackle the challenging problem of rendering real‐world 360° panorama videos that support full 6 degrees‐of‐freedom (DoF) head motion from a prerecorded omnidirectional stereo (ODS) video. In contrast to recent approaches that create novel views for individual panorama frames, we introduce a video‐specific temporally‐consistent multi‐sphere image (MSI) scene representation. Given a conventional ODS video, we first extract information by estimating framewise descriptive feature maps. Then, we optimize the global MSI model using theory from recent research on neural radiance fields. Instead of a continuous scene function, this multi‐sphere image (MSI) representation depicts colour and density information only for a discrete set of concentric spheres. To further improve the temporal consistency of our results, we apply an ancillary refinement step which optimizes the temporal coherency between successive video frames. Direct comparisons to recent baseline approaches show that our global MSI optimization yields superior performance in terms of visual quality. Our code and data will be made publicly available.
Description
@article{10.1111:cgf.14796,
journal = {Computer Graphics Forum},
title = {{Immersive Free‐Viewpoint Panorama Rendering from Omnidirectional Stereo Video}},
author = {Mühlhausen, Moritz and Kappel, Moritz and Kassubeck, Marc and Wöhler, Leslie and Grogorick, Steve and Castillo, Susana and Eisemann, Martin and Magnor, Marcus},
year = {2023},
publisher = {© 2023 Eurographics ‐ The European Association for Computer Graphics and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {10.1111/cgf.14796}
}