Search Results

Now showing 1 - 10 of 16
  • Item
    Automatic Differentiable Procedural Modeling
    (The Eurographics Association and John Wiley & Sons Ltd., 2022) Gaillard, Mathieu; Krs, Vojtech; Gori, Giorgio; Mech, Radomƭr; Benes, Bedrich; Chaine, Raphaƫlle; Kim, Min H.
    Procedural modeling allows for an automatic generation of large amounts of similar assets, but there is limited control over the generated output. We address this problem by introducing Automatic Differentiable Procedural Modeling (ADPM). The forward procedural model generates a final editable model. The user modifies the output interactively, and the modifications are transferred back to the procedural model as its parameters by solving an inverse procedural modeling problem. We present an auto-differentiable representation of the procedural model that significantly accelerates optimization. In ADPM the procedural model is always available, all changes are non-destructive, and the user can interactively model the 3D object while keeping the procedural representation. ADPM provides the user with precise control over the resulting model comparable to non-procedural interactive modeling. ADPM is node-based, and it generates hierarchical 3D scene geometry converted to a differentiable computational graph. Our formulation focuses on the differentiability of high-level primitives and bounding volumes of components of the procedural model rather than the detailed mesh geometry. Although this high-level formulation limits the expressiveness of user edits, it allows for efficient derivative computation and enables interactivity. We designed a new optimizer to solve for inverse procedural modeling. It can detect that an edit is under-determined and has degrees of freedom. Leveraging cheap derivative evaluation, it can explore the region of optimality of edits and suggest various configurations, all of which achieve the requested edit differently. We show our system's efficiency on several examples, and we validate it by a user study.
  • Item
    Precise High-order Meshing of 2D Domains with Rational BƩzier Curves
    (The Eurographics Association and John Wiley & Sons Ltd., 2022) Yang, Jinlin; Liu, Shibo; Chai, Shuangming; Liu, Ligang; Fu, Xiao-Ming; Campen, Marcel; Spagnuolo, Michela
    We propose a novel method to generate a high-order triangular mesh for an input 2D domain with two key characteristics: (1) the mesh precisely conforms to a set of input piecewise rational domain curves, and (2) the geometric map on each curved triangle is injective. Central to the algorithm is a new sufficient condition for placing control points of a rational BƩzier triangle to guarantee that the conformance and injectivity constraints are theoretically satisfied. Taking advantage of this condition, we provide an explicit construct that robustly creates higher-order 2D meshes satisfying the two characteristics. We demonstrate the robustness and effectiveness of our algorithm over a data set containing 2200 examples.
  • Item
    Generative Deformable Radiance Fields for Disentangled Image Synthesis of Topology-Varying Objects
    (The Eurographics Association and John Wiley & Sons Ltd., 2022) Wang, Ziyu; Deng, Yu; Yang, Jiaolong; Yu, Jingyi; Tong, Xin; Umetani, Nobuyuki; Wojtan, Chris; Vouga, Etienne
    3D-aware generative models have demonstrated their superb performance to generate 3D neural radiance fields (NeRF) from a collection of monocular 2D images even for topology-varying object categories. However, these methods still lack the capability to separately control the shape and appearance of the objects in the generated radiance fields. In this paper, we propose a generative model for synthesizing radiance fields of topology-varying objects with disentangled shape and appearance variations. Our method generates deformable radiance fields, which builds the dense correspondence between the density fields of the objects and encodes their appearances in a shared template field. Our disentanglement is achieved in an unsupervised manner without introducing extra labels to previous 3D-aware GAN training. We also develop an effective image inversion scheme for reconstructing the radiance field of an object in a real monocular image and manipulating its shape and appearance. Experiments show that our method can successfully learn the generative model from unstructured monocular images and well disentangle the shape and appearance for objects (e.g., chairs) with large topological variance. The model trained on synthetic data can faithfully reconstruct the real object in a given single image and achieve high-quality texture and shape editing results.
  • Item
    Closed Space-filling Curves with Controlled Orientation for 3D Printing
    (The Eurographics Association and John Wiley & Sons Ltd., 2022) Bedel, Adrien; Coudert-Osmont, Yoann; Martƭnez, JonƠs; Nishat, Rahnuma Islam; Whitesides, Sue; Lefebvre, Sylvain; Chaine, Raphaƫlle; Kim, Min H.
    We explore the optimization of closed space-filling curves under orientation objectives. By solidifying material along the closed curve, solid layers of 3D prints can be manufactured in a single continuous extrusion motion. The control over orientation enables the deposition to align with specific directions in different areas, or to produce a locally uniform distribution of orientations, patterning the solidified volume in a precisely controlled manner. Our optimization framework proceeds in two steps. First, we cast a combinatorial problem, optimizing Hamiltonian cycles within a specially constructed graph. We rely on a stochastic optimization process based on local operators that modify a cycle while preserving its Hamiltonian property. Second, we use the result to initialize a geometric optimizer that improves the smoothness and uniform coverage of the cycle while further optimizing for alignment and orientation objectives.
  • Item
    Differentiable 3D CAD Programs for Bidirectional Editing
    (The Eurographics Association and John Wiley & Sons Ltd., 2022) Cascaval, Dan; Shalah, Mira; Quinn, Phillip; Bodik, Rastislav; Agrawala, Maneesh; Schulz, Adriana; Chaine, Raphaƫlle; Kim, Min H.
    Modern CAD tools represent 3D designs not only as geometry, but also as a program composed of geometric operations, each of which depends on a set of parameters. Program representations enable meaningful and controlled shape variations via parameter changes. However, achieving desired modifications solely through parameter editing is challenging when CAD models have not been explicitly authored to expose select degrees of freedom in advance. We introduce a novel bidirectional editing system for 3D CAD programs. In addition to editing the CAD program, users can directly manipulate 3D geometry and our system infers parameter updates to keep both representations in sync. We formulate inverse edits as a set of constrained optimization objectives, returning plausible updates to program parameters that both match user intent and maintain program validity. Our approach implements an automatically differentiable domain-specific language for CAD programs, providing derivatives for this optimization to be performed quickly on any expressed program. Our system enables rapid, interactive exploration of a constrained 3D design space by allowing users to manipulate the program and geometry interchangeably during design iteration. While our approach is not designed to optimize across changes in geometric topology, we show it is expressive and performant enough for users to produce a diverse set of design variants, even when the CAD program contains a relatively large number of parameters.
  • Item
    Fabricable Multi-Scale Wang Tiles
    (The Eurographics Association and John Wiley & Sons Ltd., 2022) Liu, Xiaokang; Li, Chenran; Lu, Lin; Deussen, Oliver; Tu, Changhe; Campen, Marcel; Spagnuolo, Michela
    Wang tiles, also known as Wang dominoes, are a jigsaw puzzle system with matching edges. Due to their compactness and expressiveness in representing variations, they have become a popular tool in the procedural synthesis of textures, height fields, 3D printing and representing other large and non-repetitive data. Multi-scale tiles created from low-level tiles allow for a higher tiling efficiency, although they face the problem of combinatorial explosion. In this paper, we propose a generation method for multi-scale Wang tiles that aims at minimizing the amount of needed tiles while still resembling a tiling appearance similar to low-level tiles. Based on a set of representative multi-scale Wang tiles, we use a dynamic generation algorithm for this purpose. Our method can be used for rapid texture synthesis and image halftoning. Respecting physical constraints, our tiles are connected, lightweight, independent of the fabrication scale, able to tile larger areas with image contents and contribute to "mass customization".
  • Item
    Deep Reconstruction of 3D Smoke Densities from Artist Sketches
    (The Eurographics Association and John Wiley & Sons Ltd., 2022) Kim, Byungsoo; Huang, Xingchang; Wuelfroth, Laura; Tang, Jingwei; Cordonnier, Guillaume; Gross, Markus; Solenthaler, Barbara; Chaine, Raphaƫlle; Kim, Min H.
    Creative processes of artists often start with hand-drawn sketches illustrating an object. Pre-visualizing these keyframes is especially challenging when applied to volumetric materials such as smoke. The authored 3D density volumes must capture realistic flow details and turbulent structures, which is highly non-trivial and remains a manual and time-consuming process. We therefore present a method to compute a 3D smoke density field directly from 2D artist sketches, bridging the gap between early-stage prototyping of smoke keyframes and pre-visualization. From the sketch inputs, we compute an initial volume estimate and optimize the density iteratively with an updater CNN. Our differentiable sketcher is embedded into the end-to-end training, which results in robust reconstructions. Our training data set and sketch augmentation strategy are designed such that it enables general applicability. We evaluate the method on synthetic inputs and sketches from artists depicting both realistic smoke volumes and highly non-physical smoke shapes. The high computational performance and robustness of our method at test time allows interactive authoring sessions of volumetric density fields for rapid prototyping of ideas by novice users.
  • Item
    Facial Animation with Disentangled Identity and Motion using Transformers
    (The Eurographics Association and John Wiley & Sons Ltd., 2022) Chandran, Prashanth; Zoss, Gaspard; Gross, Markus; Gotardo, Paulo; Bradley, Derek; Dominik L. Michels; Soeren Pirk
    We propose a 3D+time framework for modeling dynamic sequences of 3D facial shapes, representing realistic non-rigid motion during a performance. Our work extends neural 3D morphable models by learning a motion manifold using a transformer architecture. More specifically, we derive a novel transformer-based autoencoder that can model and synthesize 3D geometry sequences of arbitrary length. This transformer naturally determines frame-to-frame correlations required to represent the motion manifold, via the internal self-attention mechanism. Furthermore, our method disentangles the constant facial identity from the time-varying facial expressions in a performance, using two separate codes to represent neutral identity and the performance itself within separate latent subspaces. Thus, the model represents identity-agnostic performances that can be paired with an arbitrary new identity code and fed through our new identity-modulated performance decoder; the result is a sequence of 3D meshes for the performance with the desired identity and temporal length. We demonstrate how our disentangled motion model has natural applications in performance synthesis, performance retargeting, key-frame interpolation and completion of missing data, performance denoising and retiming, and other potential applications that include full 3D body modeling.
  • Item
    Interactive Editing of Discrete Chebyshev Nets
    (The Eurographics Association and John Wiley & Sons Ltd., 2022) Li, Rui-Zeng; Guo, Jia-Peng; Wang, Qi; Chai, Shuangming; Liu, Ligang; Fu, Xiao-Ming; Chaine, Raphaƫlle; Kim, Min H.
    We propose an interactive method to edit a discrete Chebyshev net, which is a quad mesh with edges of the same length. To ensure that the edited mesh is always a discrete Chebyshev net, the maximum difference of all edge lengths should be zero during the editing process. Hence, we formulate an objective function using lp-norm (p > 2) to force the maximum length deviation to approach zero in practice. To optimize the nonlinear and non-convex objective function interactively and efficiently, we develop a novel second-order solver. The core of the solver is to construct a new convex majorizer for our objective function to achieve fast convergence. We present two acceleration strategies to further reduce the optimization time, including adaptive p change and adaptive variables reduction. A large number of experiments demonstrate the capability and feasibility of our method for interactively editing complex discrete Chebyshev nets.
  • Item
    MoCo-Flow: Neural Motion Consensus Flow for Dynamic Humans in Stationary Monocular Cameras
    (The Eurographics Association and John Wiley & Sons Ltd., 2022) Chen, Xuelin; Li, Weiyu; Cohen-Or, Daniel; Mitra, Niloy J.; Chen, Baoquan; Chaine, Raphaƫlle; Kim, Min H.
    Synthesizing novel views of dynamic humans from stationary monocular cameras is a specialized but desirable setup. This is particularly attractive as it does not require static scenes, controlled environments, or specialized capture hardware. In contrast to techniques that exploit multi-view observations, the problem of modeling a dynamic scene from a single view is significantly more under-constrained and ill-posed. In this paper, we introduce Neural Motion Consensus Flow (MoCo-Flow), a representation that models dynamic humans in stationary monocular cameras using a 4D continuous time-variant function. We learn the proposed representation by optimizing for a dynamic scene that minimizes the total rendering error, over all the observed images. At the heart of our work lies a carefully designed optimization scheme, which includes a dedicated initialization step and is constrained by a motion consensus regularization on the estimated motion flow. We extensively evaluate MoCo-Flow on several datasets that contain human motions of varying complexity, and compare, both qualitatively and quantitatively, to several baselines and ablated variations of our methods, showing the efficacy and merits of the proposed approach. Pretrained model, code, and data will be released for research purposes upon paper acceptance.