• Login
    View Item 
    •   Eurographics DL Home
    • Computer Graphics Forum
    • Volume 39 (2020)
    • 39-Issue 7
    • View Item
    •   Eurographics DL Home
    • Computer Graphics Forum
    • Volume 39 (2020)
    • 39-Issue 7
    • View Item
    JavaScript is disabled for your browser. Some features of this site may not work without it.

    InstanceFusion: Real-time Instance-level 3D Reconstruction Using a Single RGBD Camera

    Thumbnail
    View/Open
    v39i7pp433-445.pdf (10.79Mb)
    supplementary_video-final.mp4 (53.07Mb)
    Date
    2020
    Author
    Lu, Feixiang ORCID
    Peng, Haotian
    Wu, Hongyu ORCID
    Yang, Jun
    Yang, Xinhang
    Cao, Ruizhi
    Zhang, Liangjun
    Yang, Ruigang
    Zhou, Bin ORCID
    Pay-Per-View via TIB Hannover:

    Try if this item/paper is available.

    Metadata
    Show full item record
    Abstract
    We present InstanceFusion, a robust real-time system to detect, segment, and reconstruct instance-level 3D objects of indoor scenes with a hand-held RGBD camera. It combines the strengths of deep learning and traditional SLAM techniques to produce visually compelling 3D semantic models. The key success comes from our novel segmentation scheme and the efficient instancelevel data fusion, which are both implemented on GPU. Specifically, for each incoming RGBD frame, we take the advantages of the RGBD features, the 3D point cloud, and the reconstructed model to perform instance-level segmentation. The corresponding RGBD data along with the instance ID are then fused to the surfel-based models. In order to sufficiently store and update these data, we design and implement a new data structure using the OpenGL Shading Language. Experimental results show that our method advances the state-of-the-art (SOTA) methods in instance segmentation and data fusion by a big margin. In addition, our instance segmentation improves the precision of 3D reconstruction, especially in the loop closure. InstanceFusion system runs 20.5Hz on a consumer-level GPU, which supports a number of augmented reality (AR) applications (e.g., 3D model registration, virtual interaction, AR map) and robot applications (e.g., navigation, manipulation, grasping). To facilitate future research and reproduce our system more easily, the source code, data, and the trained model are released on Github: https://github.com/Fancomi2017/InstanceFusion.
    BibTeX
    @article {10.1111:cgf.14157,
    journal = {Computer Graphics Forum},
    title = {{InstanceFusion: Real-time Instance-level 3D Reconstruction Using a Single RGBD Camera}},
    author = {Lu, Feixiang and Peng, Haotian and Wu, Hongyu and Yang, Jun and Yang, Xinhang and Cao, Ruizhi and Zhang, Liangjun and Yang, Ruigang and Zhou, Bin},
    year = {2020},
    publisher = {The Eurographics Association and John Wiley & Sons Ltd.},
    ISSN = {1467-8659},
    DOI = {10.1111/cgf.14157}
    }
    URI
    https://doi.org/10.1111/cgf.14157
    https://diglib.eg.org:443/handle/10.1111/cgf14157
    Collections
    • 39-Issue 7

    Eurographics Association copyright © 2013 - 2023 
    Send Feedback | Contact - Imprint | Data Privacy Policy | Disable Google Analytics
    Theme by @mire NV
    System hosted at  Graz University of Technology.
    TUGFhA
     

     

    Browse

    All of Eurographics DLCommunities & CollectionsBy Issue DateAuthorsTitlesSubjectsThis CollectionBy Issue DateAuthorsTitlesSubjects

    My Account

    LoginRegister

    Statistics

    View Usage Statistics

    BibTeX | TOC

    Create BibTeX Create Table of Contents

    Eurographics Association copyright © 2013 - 2023 
    Send Feedback | Contact - Imprint | Data Privacy Policy | Disable Google Analytics
    Theme by @mire NV
    System hosted at  Graz University of Technology.
    TUGFhA