Light Field Video Compression and Real Time Rendering

Simple item page
dc.contributor.authorHajisharif, Saghien_US
dc.contributor.authorMiandji, Ehsanen_US
dc.contributor.authorLarsson, Peren_US
dc.contributor.authorTran, Kieten_US
dc.contributor.authorUnger, Jonasen_US
dc.contributor.editorLee, Jehee and Theobalt, Christian and Wetzstein, Gordonen_US
dc.date.accessioned2019-10-14T05:07:21Z
dc.date.available2019-10-14T05:07:21Z
dc.date.issued2019
dc.description.abstractLight field imaging is rapidly becoming an established method for generating flexible image based description of scene appearances. Compared to classical 2D imaging techniques, the angular information included in light fields enables effects such as post-capture refocusing and the exploration of the scene from different vantage points. In this paper, we describe a novel GPU pipeline for compression and real-time rendering of light field videos with full parallax. To achieve this, we employ a dictionary learning approach and train an ensemble of dictionaries capable of efficiently representing light field video data using highly sparse coefficient sets. A novel, key element in our representation is that we simultaneously compress both image data (pixel colors) and the auxiliary information (depth, disparity, or optical flow) required for view interpolation. During playback, the coefficients are streamed to the GPU where the light field and the auxiliary information are reconstructed using the dictionary ensemble and view interpolation is performed. In order to realize the pipeline we present several technical contributions including a denoising scheme enhancing the sparsity in the dataset which enables higher compression ratios, and a novel pruning strategy which reduces the size of the dictionary ensemble and leads to significant reductions in computational complexity during the encoding of a light field. Our approach is independent of the light field parameterization and can be used with data from any light field video capture system. To demonstrate the usefulness of our pipeline, we utilize various publicly available light field video datasets and discuss the medical application of documenting heart surgery.en_US
dc.description.number7
dc.description.sectionheadersComputational Photography
dc.description.seriesinformationComputer Graphics Forum
dc.description.volume38
dc.identifier.doi10.1111/cgf.13835
dc.identifier.issn1467-8659
dc.identifier.pages265-276
dc.identifier.urihttps://doi.org/10.1111/cgf.13835
dc.identifier.urihttps://diglib.eg.org:443/handle/10.1111/cgf13835
dc.publisherThe Eurographics Association and John Wiley & Sons Ltd.en_US
dc.subjectComputer graphics
dc.subjectImage
dc.subjectbased rendering
dc.subjectComputational photography
dc.subjectImage compression
dc.titleLight Field Video Compression and Real Time Renderingen_US
Files
Collections
38-Issue 7