Search Results

Now showing 1 - 10 of 20
  • Item
    Interactive Facades - Analysis and Synthesis of Semi-Regular Facades
    (The Eurographics Association and Blackwell Publishing Ltd., 2013) AlHalawani, Sawsan; Yang, Yong-Liang; Liu, Han; Mitra, Niloy J.; I. Navazo, P. Poulin
    Urban facades regularly contain interesting variations due to allowed deformations of repeated elements (e.g., windows in different open or close positions) posing challenges to state-of-the-art facade analysis algorithms. We propose a semi-automatic framework to recover both repetition patterns of the elements and their individual deformation parameters to produce a factored facade representation. Such a representation enables a range of applications including interactive facade images, improved multi-view stereo reconstruction, facade-level change detection, and novel image editing possibilities.
  • Item
    Smart Variations: Functional Substructures for Part Compatibility
    (The Eurographics Association and Blackwell Publishing Ltd., 2013) Zheng, Youyi; Cohen-Or, Daniel; Mitra, Niloy J.; I. Navazo, P. Poulin
    As collections of 3D models continue to grow, reusing model parts allows generation of novel model variations. Naïvely swapping parts across models, however, leads to implausible results, especially when mixing parts across different model families. Hence, the user has to manually ensure that the final model remains functionally valid. We claim that certain symmetric functional arrangements (SFARR-s), which are special arrangements among symmetrically related substructures, bear close relation to object functions. Hence, we propose a purely geometric approach based on such substructures to match, replace, and position triplets of parts to create non-trivial, yet functionally plausible, model variations. We demonstrate that starting even from a small set of models such a simple geometric approach can produce a diverse set of non-trivial and plausible model variations.
  • Item
    Repetition Maximization based Texture Rectification
    (The Eurographics Association and John Wiley and Sons Ltd., 2012) Aiger, Dror; Cohen-Or, Daniel; Mitra, Niloy J.; P. Cignoni and T. Ertl
    Many photographs are taken in perspective. Techniques for rectifying resulting perspective distortions typically rely on the existence of parallel lines in the scene. In scenarios where such parallel lines are hard to automatically extract or manually annotate, the unwarping process remains a challenge. In this paper, we introduce an automatic algorithm to rectifying images containing textures of repeated elements lying on an unknown plane. We unwrap the input by maximizing for image self-similarity over the space of homography transformations. We map a set of detected regional descriptors to surfaces in a transformation space, compute the intersection points among triplets of such surfaces, and then use consensus among the projected intersection points to extract the correcting transform. Our algorithm is global, robust, and does not require explicit or accurate detection of similar elements. We evaluate our method on a variety of challenging textures and images. The rectified outputs are directly useful for various tasks including texture synthesis, image completion, etc.
  • Item
    Interactive Videos: Plausible Video Editing using Sparse Structure Points
    (The Eurographics Association and John Wiley & Sons Ltd., 2016) Chang, Chia-Sheng; Chu, Hung-Kuo; Mitra, Niloy J.; Joaquim Jorge and Ming Lin
    Video remains the method of choice for capturing temporal events. However, without access to the underlying 3D scene models, it remains difficult to make object level edits in a single video or across multiple videos. While it may be possible to explicitly reconstruct the 3D geometries to facilitate these edits, such a workflow is cumbersome, expensive, and tedious. In this work, we present a much simpler workflow to create plausible editing and mixing of raw video footage using only sparse structure points (SSP) directly recovered from the raw sequences. First, we utilize user-scribbles to structure the point representations obtained using structure-from-motion on the input videos. The resultant structure points, even when noisy and sparse, are then used to enable various video edits in 3D, including view perturbation, keyframe animation, object duplication and transfer across videos, etc. Specifically, we describe how to synthesize object images from new views adopting a novel image-based rendering technique using the SSPs as proxy for the missing 3D scene information. We propose a structure-preserving image warping on multiple input frames adaptively selected from object video, followed by a spatio-temporally coherent image stitching to compose the final object image. Simple planar shadows and depth maps are synthesized for objects to generate plausible video sequence mimicking real-world interactions. We demonstrate our system on a variety of input videos to produce complex edits, which are otherwise difficult to achieve.
  • Item
    Neurosymbolic Models for Computer Graphics
    (The Eurographics Association and John Wiley & Sons Ltd., 2023) Ritchie, Daniel; Guerrero, Paul; Jones, R. Kenny; Mitra, Niloy J.; Schulz, Adriana; Willis, Karl D. D.; Wu, Jiajun; Bousseau, Adrien; Theobalt, Christian
    Procedural models (i.e. symbolic programs that output visual data) are a historically-popular method for representing graphics content: vegetation, buildings, textures, etc. They offer many advantages: interpretable design parameters, stochastic variations, high-quality outputs, compact representation, and more. But they also have some limitations, such as the difficulty of authoring a procedural model from scratch. More recently, AI-based methods, and especially neural networks, have become popular for creating graphic content. These techniques allow users to directly specify desired properties of the artifact they want to create (via examples, constraints, or objectives), while a search, optimization, or learning algorithm takes care of the details. However, this ease of use comes at a cost, as it's often hard to interpret or manipulate these representations. In this state-of-the-art report, we summarize research on neurosymbolic models in computer graphics: methods that combine the strengths of both AI and symbolic programs to represent, generate, and manipulate visual data. We survey recent work applying these techniques to represent 2D shapes, 3D shapes, and materials & textures. Along the way, we situate each prior work in a unified design space for neurosymbolic models, which helps reveal underexplored areas and opportunities for future research.
  • Item
    Recurring Part Arrangements in Shape Collections
    (The Eurographics Association and John Wiley and Sons Ltd., 2014) Zheng, Youyi; Cohen-Or, Daniel; Averkiou, Melinos; Mitra, Niloy J.; B. Levy and J. Kautz
    Extracting semantically related parts across models remains challenging, especially without supervision. The common approach is to co-analyze a model collection, while assuming the existence of descriptive geometric features that can directly identify related parts. In the presence of large shape variations, common geometric features, however, are no longer sufficiently descriptive. In this paper, we explore an indirect top-down approach, where instead of part geometry, part arrangements extracted from each model are compared. The key observation is that while a direct comparison of part geometry can be ambiguous, part arrangements, being higher level structures, remain consistent, and hence can be used to discover latent commonalities among semantically related shapes. We show that our indirect analysis leads to the detection of recurring arrangements of parts, which are otherwise difficult to discover in a direct unsupervised setting. We evaluate our algorithm on ground truth datasets and report advantages over geometric similarity-based bottom-up co-segmentation algorithms.
  • Item
    Variation-Factored Encoding of Facade Images
    (The Eurographics Association, 2012) Alsisan, Suhib; Mitra, Niloy J.; Carlos Andujar and Enrico Puppo
    Urban facades contain large-scale repetitions in the form of windows, doors, etc. Such elements often are in different configurations (e.g., open or closed) obscuring their regular arrangements to any direct low-level pixel matching based repetition detection.We propose a variation-factored representation for facade images by progressively favoring larger repeated structures while allowing relabeling using candidate element types. We formulate the problem as a Markov Random Field (MRF) based optimization, and evaluate the algorithm on a large number of benchmark facade images. Such a facade encoding is very compact and can be used for rapid generation of realistic 3D models with variations suitable for online map viewers or mobile navigation aids.
  • Item
    Factored Facade Acquisition using Symmetric Line Arrangements
    (The Eurographics Association and John Wiley and Sons Ltd., 2012) Ceylan, Duygu; Mitra, Niloy J.; Li, Hao; Weise, Thibaut; Pauly, Mark; P. Cignoni and T. Ertl
    We introduce a novel framework for image-based 3D reconstruction of urban buildings based on symmetry priors. Starting from image-level edges, we generate a sparse and approximate set of consistent 3D lines. These lines are then used to simultaneously detect symmetric line arrangements while refining the estimated 3D model. Operating both on 2D image data and intermediate 3D feature representations, we perform iterative feature consolidation and effective outlier pruning, thus eliminating reconstruction artifacts arising from ambiguous or wrong stereo matches. We exploit non-local coherence of symmetric elements to generate precise model reconstructions, even in the presence of a significant amount of outlier image-edges arising from reflections, shadows, outlier objects, etc. We evaluate our algorithm on several challenging test scenarios, both synthetic and real. Beyond reconstruction, the extracted symmetry patterns are useful towards interactive and intuitive model manipulations.
  • Item
    Graph Abstraction for Simplified Proofreading of Slice-based Volume Segmentation
    (The Eurographics Association, 2013) Sicat, Ronell B.; Hadwiger, Markus; Mitra, Niloy J.; M.- A. Otaduy and O. Sorkine
    Volume segmentation is an integral data analysis tool in experimental science. For example, in neuroscience, analysis of 3D volumes of neural structures from electron microscopy data is a key analysis step. Despite advances in computational methods, experts still prefer to manually proofread and correct the automatic segmentation outputs. Such corrections are often annotated at the level of data slices in order to minimize distortion artifacts and effectively handle the massive data volumes. In absence of crucial global context in 3D, such a workflow remains tedious, time consuming, and error prone. In this paper, we present a simple graph-based abstraction for segmentation volumes leading to an interactive proofreading tool making the process simpler, faster, and intuitive. Starting from an initial volume segmentation, we first construct a graph abstraction and then use it to identify potential problematic regions for the user to investigate and correct spurious segmentations, if identified. We also use the graph to suggest automatic corrections, thus drastically simplifying the proofreading effort. We implemented the proofreading tool as an Avizo c plugin and evaluated the method on complex real-world use cases.
  • Item
    Deep Learning for Graphics
    (The Eurographics Association, 2018) Mitra, Niloy J.; Ritschel, Tobias; Kokkinos, Iasonas; Guerrero, Paul; Kim, Vladimir; Rematas, Konstantinos; Yumer, Ersin; Ritschel, Tobias and Telea, Alexandru
    In computer graphics, many traditional problems are now better handled by deep-learning based data-driven methods. In applications that operate on regular 2D domains, like image processing and computational photography, deep networks are state-of-the-art, beating dedicated hand-crafted methods by significant margins. More recently, other domains such as geometry processing, animation, video processing, and physical simulations have benefited from deep learning methods as well. The massive volume of research that has emerged in just a few years is often difficult to grasp for researchers new to this area. This tutorial gives an organized overview of core theory, practice, and graphics-related applications of deep learning.