Search Results

Now showing 1 - 10 of 14
  • Item
    Towards a Neural Graphics Pipeline for Controllable Image Generation
    (The Eurographics Association and John Wiley & Sons Ltd., 2021) Chen, Xuelin; Cohen-Or, Daniel; Chen, Baoquan; Mitra, Niloy J.; Mitra, Niloy and Viola, Ivan
    In this paper, we leverage advances in neural networks towards forming a neural rendering for controllable image generation, and thereby bypassing the need for detailed modeling in conventional graphics pipeline. To this end, we present Neural Graphics Pipeline (NGP), a hybrid generative model that brings together neural and traditional image formation models. NGP decomposes the image into a set of interpretable appearance feature maps, uncovering direct control handles for controllable image generation. To form an image, NGP generates coarse 3D models that are fed into neural rendering modules to produce view-specific interpretable 2D maps, which are then composited into the final output image using a traditional image formation model. Our approach offers control over image generation by providing direct handles controlling illumination and camera parameters, in addition to control over shape and appearance variations. The key challenge is to learn these controls through unsupervised training that links generated coarse 3D models with unpaired real images via neural and traditional (e.g., Blinn- Phong) rendering functions, without establishing an explicit correspondence between them. We demonstrate the effectiveness of our approach on controllable image generation of single-object scenes. We evaluate our hybrid modeling framework, compare with neural-only generation methods (namely, DCGAN, LSGAN, WGAN-GP, VON, and SRNs), report improvement in FID scores against real images, and demonstrate that NGP supports direct controls common in traditional forward rendering. Code is available at http://geometry.cs.ucl.ac.uk/projects/2021/ngp.
  • Item
    Moving Basis Decomposition for Precomputed Light Transport
    (The Eurographics Association and John Wiley & Sons Ltd., 2021) Silvennoinen, Ari; Sloan, Peter-Pike; Bousseau, Adrien and McGuire, Morgan
    We study the problem of efficient representation of potentially high-dimensional, spatially coherent signals in the context of precomputed light transport. We present a basis decomposition framework, Moving Basis Decomposition (MBD), that generalizes many existing basis expansion methods and enables high-performance, seamless reconstruction of compressed data. We develop an algorithm for solving large-scale MBD problems. We evaluate MBD against state-of-the-art in a series of controlled experiments and describe a real-world application, where MBD serves as the backbone of a scalable global illumination system powering multiple, current and upcoming 60Hz AAA-titles running on a wide range of hardware platforms.
  • Item
    Correlation-Aware Multiple Importance Sampling for Bidirectional Rendering Algorithms
    (The Eurographics Association and John Wiley & Sons Ltd., 2021) Grittmann, Pascal; Georgiev, Iliyan; Slusallek, Philipp; Mitra, Niloy and Viola, Ivan
    Combining diverse sampling techniques via multiple importance sampling (MIS) is key to achieving robustness in modern Monte Carlo light transport simulation. Many such methods additionally employ correlated path sampling to boost efficiency. Photon mapping, bidirectional path tracing, and path-reuse algorithms construct sets of paths that share a common prefix. This correlation is ignored by classical MIS heuristics, which can result in poor technique combination and noisy images.We propose a practical and robust solution to that problem. Our idea is to incorporate correlation knowledge into the balance heuristic, based on known path densities that are already required for MIS. This correlation-aware heuristic can achieve considerably lower error than the balance heuristic, while avoiding computational and memory overhead.
  • Item
    Temporally Reliable Motion Vectors for Real-time Ray Tracing
    (The Eurographics Association and John Wiley & Sons Ltd., 2021) Zeng, Zheng; Liu, Shiqiu; Yang, Jinglei; Wang, Lu; Yan, Ling-Qi; Mitra, Niloy and Viola, Ivan
    Real-time ray tracing (RTRT) is being pervasively applied. The key to RTRT is a reliable denoising scheme that reconstructs clean images from significantly undersampled noisy inputs, usually at 1 sample per pixel as limited by current hardware's computing power. The state of the art reconstruction methods all rely on temporal filtering to find correspondences of current pixels in the previous frame, described using per-pixel screen-space motion vectors. While these approaches are demonstrated powerful, they suffer from a common issue that the temporal information cannot be used when the motion vectors are not valid, i.e. when temporal correspondences are not obviously available or do not exist in theory. We introduce temporally reliable motion vectors that aim at deeper exploration of temporal coherence, especially for the generally-believed difficult applications on shadows, glossy reflections and occlusions, with the key idea to detect and track the cause of each effect. We show that our temporally reliable motion vectors produce significantly better temporal results on a variety of dynamic scenes when compared to the state of the art methods, but with negligible performance overhead.
  • Item
    A Multiscale Microfacet Model Based on Inverse Bin Mapping
    (The Eurographics Association and John Wiley & Sons Ltd., 2021) Atanasov, Asen; Wilkie, Alexander; Koylazov, Vladimir; Krivánek, Jaroslav; Mitra, Niloy and Viola, Ivan
    Accurately controllable shading detail is a crucial aspect of realistic appearance modelling. Two fundamental building blocks for this are microfacet BRDFs, which describe the statistical behaviour of infinitely small facets, and normal maps, which provide user-controllable spatio-directional surface features. We analyse the filtering of the combined effect of a microfacet BRDF and a normal map. By partitioning the half-vector domain into bins we show that the filtering problem can be reduced to evaluation of an integral histogram (IH), a generalization of a summed-area table (SAT). Integral histograms are known for their large memory requirements, which are usually proportional to the number of bins. To alleviate this, we introduce Inverse Bin Maps, a specialised form of IH with a memory footprint that is practically independent of the number of bins. Based on these, we present a memory-efficient, production-ready approach for filtering of high resolution normal maps with arbitrary Beckmann flake roughness. In the corner case of specular normal maps (zero, or very small roughness values) our method shows similar convergence rates to the current state of the art, and is also more memory efficient.
  • Item
    SnakeBinning: Efficient Temporally Coherent Triangle Packing for Shading Streaming
    (The Eurographics Association and John Wiley & Sons Ltd., 2021) Hladky, Jozef; Seidel, Hans-Peter; Steinberger, Markus; Mitra, Niloy and Viola, Ivan
    Streaming rendering, e.g., rendering in the cloud and streaming via a mobile connection, suffers from increased latency and unreliable connections. High quality framerate upsampling can hide these issues, especially when capturing shading into an atlas and transmitting it alongside geometric information. The captured shading information must consider triangle footprints and temporal stability to ensure efficient video encoding. Previous approaches only consider either temporal stability or sample distributions, but none focuses on both. With SnakeBinning, we present an efficient triangle packing approach that adjusts sample distributions and caters for temporal coherence. Using a multi-dimensional binning approach, we enforce tight packing among triangles while creating optimal sample distributions. Our binning is built on top of hardware supported real-time rendering where bins are mapped to individual pixels in a virtual framebuffer. Fragment shader interlock and atomic operations enforce global ordering of triangles within each bin, and thus temporal coherence according to the primitive order is achieved. Resampling the bin distribution guarantees high occupancy among all bins and a dense atlas packing. Shading samples are directly captured into the atlas using a rasterization pass, adjusting samples for perspective effects and creating a tight packing. Comparison to previous atlas packing approaches shows that our approach is faster than previous work and achieves the best sample distributions while maintaining temporal coherence. In this way, SnakeBinning achieves the highest rendering quality under equal atlas memory requirements. At the same time, its temporal coherence ensures that we require equal or less bandwidth than previous state-of-the-art. As SnakeBinning outperforms previous approach in all relevant aspects, it is the preferred choice for texture-based streaming rendering.
  • Item
    Thin-Volume Visualization on Curved Domains
    (The Eurographics Association and John Wiley & Sons Ltd., 2021) Herter, Felix; Hege, Hans-Christian; Hadwiger, Markus; Lepper, Verena; Baum, Daniel; Borgo, Rita and Marai, G. Elisabeta and Landesberger, Tatiana von
    Thin, curved structures occur in many volumetric datasets. Their analysis using classical volume rendering is difficult because parts of such structures can bend away or hide behind occluding elements. This problem cannot be fully compensated by effective navigation alone, as structure-adapted navigation in the volume is cumbersome and only parts of the structure are visible in each view. We solve this problem by rendering a spatially transformed view of the volume so that an unobstructed visualization of the entire curved structure is obtained. As a result, simple and intuitive navigation becomes possible. The domain of the spatial transform is defined by a triangle mesh that is topologically equivalent to an open disc and that approximates the structure of interest. The rendering is based on ray-casting, in which the rays traverse the original volume. In order to carve out volumes of varying thicknesses, the lengths of the rays as well as the positions of the mesh vertices can be easily modified by interactive painting under view control. We describe a prototypical implementation and demonstrate the interactive visual inspection of complex structures from digital humanities, biology, medicine, and material sciences. The visual representation of the structure as a whole allows for easy inspection of interesting substructures in their original spatial context. Overall, we show that thin, curved structures in volumetric data can be excellently visualized using ray-casting-based volume rendering of transformed views defined by guiding surface meshes, supplemented by interactive, local modifications of ray lengths and vertex positions.
  • Item
    Z-Thickness Blending: Effective Fragment Merging for Multi-Fragment Rendering
    (The Eurographics Association and John Wiley & Sons Ltd., 2021) Kim, Dongjoon; Kye, Heewon; Zhang, Fang-Lue and Eisemann, Elmar and Singh, Karan
    An effective fragment merging technique is presented in this study that addresses multi-fragment problems, including fragment overflow and z-fighting, and provides visual effects that are beneficial for various screen-space rendering algorithms. The proposed method merges locally adjacent fragments along the viewing direction to resolve the aforementioned problems based on cost-effective multi-layer representation and coplanar blending. We introduce a z-thickness model based on the radiosity spreading from the viewing z-direction. Moreover, we present the fragment-merging schemes and rules for determining the visibility of the merged fragments based on the proposed z-thickness model. The proposed method is targeted at multi-fragment rendering that handles individual fragments (e.g., k-buffer) instead of representing the fragments as an approximated transmittance function. In addition, our method provides a smooth visibility transition across overlapping fragments, resulting in visual advantages in various visualization applications. In this paper, we demonstrate the advantages of the proposed method through several screen-space rendering applications.
  • Item
    Cyclostationary Gaussian Noise: Theory and Synthesis
    (The Eurographics Association and John Wiley & Sons Ltd., 2021) Lutz, Nicolas; Sauvage, Basile; Dischler, Jean-Michel; Mitra, Niloy and Viola, Ivan
    Stationary Gaussian processes have been used for decades in the context of procedural noises to model and synthesize textures with no spatial organization. In this paper we investigate cyclostationary Gaussian processes, whose statistics are repeated periodically. It enables the modeling of noises having periodic spatial variations, which we call "cyclostationary Gaussian noises". We adapt to the cyclostationary context several stationary noises along with their synthesis algorithms: spot noise, Gabor noise, local random-phase noise, high-performance noise, and phasor noise. We exhibit real-time synthesis of a variety of visual patterns having periodic spatial variations.
  • Item
    Rank-1 Lattices for Efficient Path Integral Estimation
    (The Eurographics Association and John Wiley & Sons Ltd., 2021) Liu, Hongli; Han, Honglei; Jiang, Min; Mitra, Niloy and Viola, Ivan
    We introduce rank-1 lattices as a quasi-random sequence to the numerical estimation of the high-dimensional path integral. Previous attempts at utilizing rank-1 lattices in computer graphics were very limited to low-dimensional applications, intentionally avoiding high dimensionality due to that the lattice search is NP-hard. We propose a novel framework that tackles this challenge, which was inspired by the rippling effect of the sample paths. Contrary to the conventional search approaches, our framework is based on recursively permuting the preliminarily selected components of the generator vector to achieve better pairwise projections and minimize the discrepancy of the path vertex coordinates in scene manifold spaces, resulting in improved rendering quality. It allows for the offline search of arbitrarily high-dimensional lattices to finish in a reasonable amount of time while removing the need to use all lattice points in the traditional definition, which opens the gate for their use in progressive rendering. Our rank-1 lattices successfully maintain the pixel variance at a comparable or even lower level compared to Sobol0 sampler, which offers a brand new solution to design efficient samplers for path tracing.