• Login
    View Item 
    •   Eurographics DL Home
    • Computer Graphics Forum
    • Volume 40 (2021)
    • 40-Issue 2
    • View Item
    •   Eurographics DL Home
    • Computer Graphics Forum
    • Volume 40 (2021)
    • 40-Issue 2
    • View Item
    JavaScript is disabled for your browser. Some features of this site may not work without it.

    Interactive Photo Editing on Smartphones via Intrinsic Decomposition

    Thumbnail
    View/Open
    v40i2pp497-510.pdf (72.54Mb)
    paper_1041_supplementary.pdf (216.8Mb)
    Projekt Deal version (72.62Mb)
    Date
    2021
    Author
    Shekhar, Sumit ORCID
    Reimann, Max ORCID
    Mayer, Maximilian
    Semmo, Amir ORCID
    Pasewaldt, Sebastian
    Döllner, Jürgen
    Trapp, Matthias ORCID
    Pay-Per-View via TIB Hannover:

    Try if this item/paper is available.

    Metadata
    Show full item record
    Abstract
    Intrinsic decomposition refers to the problem of estimating scene characteristics, such as albedo and shading, when one view or multiple views of a scene are provided. The inverse problem setting, where multiple unknowns are solved given a single known pixel-value, is highly under-constrained. When provided with correlating image and depth data, intrinsic scene decomposition can be facilitated using depth-based priors, which nowadays is easy to acquire with high-end smartphones by utilizing their depth sensors. In this work, we present a system for intrinsic decomposition of RGB-D images on smartphones and the algorithmic as well as design choices therein. Unlike state-of-the-art methods that assume only diffuse reflectance, we consider both diffuse and specular pixels. For this purpose, we present a novel specularity extraction algorithm based on a multi-scale intensity decomposition and chroma inpainting. At this, the diffuse component is further decomposed into albedo and shading components. We use an inertial proximal algorithm for non-convex optimization (iPiano) to ensure albedo sparsity. Our GPUbased visual processing is implemented on iOS via the Metal API and enables interactive performance on an iPhone 11 Pro. Further, a qualitative evaluation shows that we are able to obtain high-quality outputs. Furthermore, our proposed approach for specularity removal outperforms state-of-the-art approaches for real-world images, while our albedo and shading layer decomposition is faster than the prior work at a comparable output quality. Manifold applications such as recoloring, retexturing, relighting, appearance editing, and stylization are shown, each using the intrinsic layers obtained with our method and/or the corresponding depth data.
    BibTeX
    @article {10.1111:cgf.142650,
    journal = {Computer Graphics Forum},
    title = {{Interactive Photo Editing on Smartphones via Intrinsic Decomposition}},
    author = {Shekhar, Sumit and Reimann, Max and Mayer, Maximilian and Semmo, Amir and Pasewaldt, Sebastian and Döllner, Jürgen and Trapp, Matthias},
    year = {2021},
    publisher = {The Eurographics Association and John Wiley & Sons Ltd.},
    ISSN = {1467-8659},
    DOI = {10.1111/cgf.142650}
    }
    URI
    https://doi.org/10.1111/cgf.142650
    https://diglib.eg.org:443/handle/10.1111/cgf142650
    Collections
    • 40-Issue 2

    Eurographics Association copyright © 2013 - 2022 
    Send Feedback | Contact - Imprint | Data Privacy Policy | Disable Google Analytics
    Theme by @mire NV
    System hosted at  Graz University of Technology.
    TUGFhA
     

     

    Browse

    All of Eurographics DLCommunities & CollectionsBy Issue DateAuthorsTitlesSubjectsThis CollectionBy Issue DateAuthorsTitlesSubjects

    My Account

    LoginRegister

    BibTeX | TOC

    Create BibTeX Create Table of Contents

    Eurographics Association copyright © 2013 - 2022 
    Send Feedback | Contact - Imprint | Data Privacy Policy | Disable Google Analytics
    Theme by @mire NV
    System hosted at  Graz University of Technology.
    TUGFhA