DSpace Repository

Search Results

Now showing 1 - 10 of 24

Frequency-Aware Reconstruction of Fluid Simulations with Generative Networks
(The Eurographics Association, 2020) Biland, Simon; Azevedo, Vinicius C.; Kim, Byungsoo; Solenthaler, Barbara; Wilkie, Alexander and Banterle, Francesco
Convolutional neural networks were recently employed to fully reconstruct fluid simulation data from a set of reduced parameters. However, since (de-)convolutions traditionally trained with supervised l1-loss functions do not discriminate between low and high frequencies in the data, the error is not minimized efficiently for higher bands. This directly correlates with the quality of the perceived results, since missing high frequency details are easily noticeable. In this paper, we analyze the reconstruction quality of generative networks and present a frequency-aware loss function that is able to focus on specific bands of the dataset during training time. We show that our approach improves reconstruction quality of fluid simulation data in mid-frequency bands, yielding perceptually better results while requiring comparable training time.
Neural Smoke Stylization with Color Transfer
(The Eurographics Association, 2020) Christen, Fabienne; Kim, Byungsoo; Azevedo, Vinicius C.; Solenthaler, Barbara; Wilkie, Alexander and Banterle, Francesco
Artistically controlling fluid simulations requires a large amount of manual work by an artist. The recently presented transportbased neural style transfer approach simplifies workflows as it transfers the style of arbitrary input images onto 3D smoke simulations. However, the method only modifies the shape of the fluid but omits color information. In this work, we therefore extend the previous approach to obtain a complete pipeline for transferring shape and color information onto 2D and 3D smoke simulations with neural networks. Our results demonstrate that our method successfully transfers colored style features consistently in space and time to smoke data for different input textures.
Neural Denoising for Spectral Monte Carlo Rendering
(The Eurographics Association, 2022) Rouphael, Robin; Noizet, Mathieu; Prévost, Stéphanie; Deleau, Hervé; Steffenel, Luiz-Angelo; Lucas, Laurent; Sauvage, Basile; Hasic-Telalovic, Jasminka
Spectral Monte Carlo (MC) rendering is still to be largely adopted partially due to the specific noise, called color noise, induced by wavelength-dependent phenomenons. Motivated by the recent advances in Monte Carlo noise reduction using Deep Learning, we propose to apply the same approach to color noise. Our implementation and training managed to reconstruct a noise-free output while conserving high-frequency details despite a loss of contrast. To address this issue, we designed a three-step pipeline using the contribution of a secondary denoiser to obtain high-quality results.
Data-driven Garment Pattern Estimation from 3D Geometries
(The Eurographics Association, 2021) Goto, Chihiro; Umetani, Nobuyuki; Theisel, Holger and Wimmer, Michael
Three-dimensional scanning technology recently becomes widely available to the public. However, it is difficult to simulate clothing deformation from the scanned people because scanned data lacks information required for the clothing simulation. In this paper, we present a technique to estimate clothing patterns from a scanned person in cloth. Our technique uses image-based deep learning to estimate the type of pattern on the projected image. The key contribution is converting image-based inference into three-dimensional clothing pattern estimation. We evaluate our technique by applying our technique to an actual scan.
State-of-the-Art in the Architecture, Methods and Applications of StyleGAN
(The Eurographics Association and John Wiley & Sons Ltd., 2022) Bermano, Amit Haim; Gal, Rinon; Alaluf, Yuval; Mokady, Ron; Nitzan, Yotam; Tov, Omer; Patashnik, Or; Cohen-Or, Daniel; Meneveaux, Daniel; Patanè, Giuseppe
Generative Adversarial Networks (GANs) have established themselves as a prevalent approach to image synthesis. Of these, StyleGAN offers a fascinating case study, owing to its remarkable visual quality and an ability to support a large array of downstream tasks. This state-of-the-art report covers the StyleGAN architecture, and the ways it has been employed since its conception, while also analyzing its severe limitations. It aims to be of use for both newcomers, who wish to get a grasp of the field, and for more experienced readers that might benefit from seeing current research trends and existing tools laid out. Among StyleGAN's most interesting aspects is its learned latent space. Despite being learned with no supervision, it is surprisingly well-behaved and remarkably disentangled. Combined with StyleGAN's visual quality, these properties gave rise to unparalleled editing capabilities. However, the control offered by StyleGAN is inherently limited to the generator's learned distribution, and can only be applied to images generated by StyleGAN itself. Seeking to bring StyleGAN's latent control to real-world scenarios, the study of GAN inversion and latent space embedding has quickly gained in popularity. Meanwhile, this same study has helped shed light on the inner workings and limitations of StyleGAN. We map out StyleGAN's impressive story through these investigations, and discuss the details that have made StyleGAN the go-to generator. We further elaborate on the visual priors StyleGAN constructs, and discuss their use in downstream discriminative tasks. Looking forward, we point out StyleGAN's limitations and speculate on current trends and promising directions for future research, such as task and target specific fine-tuning.
Towards L-System Captioning for Tree Reconstruction
(The Eurographics Association, 2023) Magnusson, Jannes S.; Hilsmann, Anna; Eisert, Peter; Babaei, Vahid; Skouras, Melina
This work proposes a novel concept for tree and plant reconstruction by directly inferring a Lindenmayer-System (L-System) word representation from image data in an image captioning approach. We train a model end-to-end which is able to translate given images into L-System words as a description of the displayed tree. To prove this concept, we demonstrate the applicability on 2D tree topologies. Transferred to real image data, this novel idea could lead to more efficient, accurate and semantically meaningful tree and plant reconstruction without using error-prone point cloud extraction, and other processes usually utilized in tree reconstruction. Furthermore, this approach bypasses the need for a predefined L-System grammar and enables species-specific L-System inference without biological knowledge.
Neurosymbolic Models for Computer Graphics
(The Eurographics Association and John Wiley & Sons Ltd., 2023) Ritchie, Daniel; Guerrero, Paul; Jones, R. Kenny; Mitra, Niloy J.; Schulz, Adriana; Willis, Karl D. D.; Wu, Jiajun; Bousseau, Adrien; Theobalt, Christian
Procedural models (i.e. symbolic programs that output visual data) are a historically-popular method for representing graphics content: vegetation, buildings, textures, etc. They offer many advantages: interpretable design parameters, stochastic variations, high-quality outputs, compact representation, and more. But they also have some limitations, such as the difficulty of authoring a procedural model from scratch. More recently, AI-based methods, and especially neural networks, have become popular for creating graphic content. These techniques allow users to directly specify desired properties of the artifact they want to create (via examples, constraints, or objectives), while a search, optimization, or learning algorithm takes care of the details. However, this ease of use comes at a cost, as it's often hard to interpret or manipulate these representations. In this state-of-the-art report, we summarize research on neurosymbolic models in computer graphics: methods that combine the strengths of both AI and symbolic programs to represent, generate, and manipulate visual data. We survey recent work applying these techniques to represent 2D shapes, 3D shapes, and materials & textures. Along the way, we situate each prior work in a unified design space for neurosymbolic models, which helps reveal underexplored areas and opportunities for future research.
State of the Art on Diffusion Models for Visual Computing
(The Eurographics Association and John Wiley & Sons Ltd., 2024) Po, Ryan; Yifan, Wang; Golyanik, Vladislav; Aberman, Kfir; Barron, Jon T.; Bermano, Amit; Chan, Eric; Dekel, Tali; Holynski, Aleksander; Kanazawa, Angjoo; Liu, C. Karen; Liu, Lingjie; Mildenhall, Ben; Nießner, Matthias; Ommer, Björn; Theobalt, Christian; Wonka, Peter; Wetzstein, Gordon; Aristidou, Andreas; Macdonnell, Rachel
The field of visual computing is rapidly advancing due to the emergence of generative artificial intelligence (AI), which unlocks unprecedented capabilities for the generation, editing, and reconstruction of images, videos, and 3D scenes. In these domains, diffusion models are the generative AI architecture of choice. Within the last year alone, the literature on diffusion-based tools and applications has seen exponential growth and relevant papers are published across the computer graphics, computer vision, and AI communities with new works appearing daily on arXiv. This rapid growth of the field makes it difficult to keep up with all recent developments. The goal of this state-of-the-art report (STAR) is to introduce the basic mathematical concepts of diffusion models, implementation details and design choices of the popular Stable Diffusion model, as well as overview important aspects of these generative AI tools, including personalization, conditioning, inversion, among others. Moreover, we give a comprehensive overview of the rapidly growing literature on diffusion-based generation and editing, categorized by the type of generated medium, including 2D images, videos, 3D objects, locomotion, and 4D scenes. Finally, we discuss available datasets, metrics, open challenges, and social implications. This STAR provides an intuitive starting point to explore this exciting topic for researchers, artists, and practitioners alike.
Adversarial Generation of Continuous Implicit Shape Representations
(The Eurographics Association, 2020) Kleineberg, Marian; Fey, Matthias; Weichert, Frank; Wilkie, Alexander and Banterle, Francesco
This work presents a generative adversarial architecture for generating three-dimensional shapes based on signed distance representations. While the deep generation of shapes has been mostly tackled by voxel and surface point cloud approaches, our generator learns to approximate the signed distance for any point in space given prior latent information. Although structurally similar to generative point cloud approaches, this formulation can be evaluated with arbitrary point density during inference, leading to fine-grained details in generated outputs. Furthermore, we study the effects of using either progressively growing voxel- or point-processing networks as discriminators, and propose a refinement scheme to strengthen the generator's capabilities in modeling the zero iso-surface decision boundary of shapes. We train our approach on the SHAPENET benchmark dataset and validate, both quantitatively and qualitatively, its performance in generating realistic 3D shapes.
Deep Learning for Graphics
(The Eurographics Association, 2018) Mitra, Niloy J.; Ritschel, Tobias; Kokkinos, Iasonas; Guerrero, Paul; Kim, Vladimir; Rematas, Konstantinos; Yumer, Ersin; Ritschel, Tobias and Telea, Alexandru
In computer graphics, many traditional problems are now better handled by deep-learning based data-driven methods. In applications that operate on regular 2D domains, like image processing and computational photography, deep networks are state-of-the-art, beating dedicated hand-crafted methods by significant margins. More recently, other domains such as geometry processing, animation, video processing, and physical simulations have benefited from deep learning methods as well. The massive volume of research that has emerged in just a few years is often difficult to grasp for researchers new to this area. This tutorial gives an organized overview of core theory, practice, and graphics-related applications of deep learning.

Filters

Author

Subject

Date

Has files

Settings

Sort By

Results per page

Search Results