Structure-aware content creation: Detection, retargeting and deformation
(2017-01-20) Wu, Xiaokun
Nowadays, access to digital information has become ubiquitous, while three-dimensional visual representation is becoming indispensable to knowledge understanding and information retrieval. Three-dimensional digitization plays a natural role in bridging connections between the real and virtual world, which prompt the huge demand for massive three-dimensional digital content. But reducing the effort required for three-dimensional modeling has been a practical problem, and long standing challenge in compute graphics and related fields. In this thesis, we propose several techniques for lightening up the content creation process, which have the common theme of being structure-aware, i.e. maintaining global relations among the parts of shape. We are especially interested in formulating our algorithms such that they make use of symmetry structures, because of their concise yet highly abstract principles are universally applicable to most regular patterns. We introduce our work from three different aspects in this thesis. First, we characterized spaces of symmetry preserving deformations, and developed a method to explore this space in real-time, which significantly simplified the generation of symmetry preserving shape variants. Second, we empirically studied three-dimensional offset statistics, and developed a fully automatic retargeting application, which is based on verified sparsity. Finally, we made step forward in solving the approximate three-dimensional partial symmetry detection problem, using a novel co-occurrence analysis method, which could serve as the foundation to high-level applications.
High-quality Face Capture, Animation and Editing from Monocular Video
(2017-06-26) Garrido, Pablo
Digitization of virtual faces in movies requires complex capture setups and extensive manual work to produce superb animations and video-realistic editing. This thesis pushes the boundaries of the digitization pipeline by proposing automatic algorithms for high-quality 3D face capture and animation, as well as photo-realistic face editing. These algorithms reconstruct and modify faces in 2D videos recorded in uncontrolled scenarios and illumination. In particular, advances in three main areas offer solutions for the lack of depth and overall uncertainty in video recordings. First, contributions in capture include model-based reconstruction of detailed, dynamic 3D geometry that exploits optical and shading cues, multilayer parametric reconstruction of accurate 3D models in unconstrained setups based on inverse rendering, and regression-based 3D lip shape enhancement from high-quality data. Second, advances in animation are video-based face reenactment based on robust appearance metrics and temporal clustering, performance-driven retargeting of detailed facial models in sync with audio, and the automatic creation of personalized controllable 3D rigs. Finally, advances in plausible photo-realistic editing are dense face albedo capture and mouth interior synthesis using image warping and 3D teeth proxies. High-quality results attained on challenging application scenarios confirm the contributions and show great potential for the automatic creation of photo-realistic 3D faces.
Physics-based Reconstruction and Animation of Humans
(Ecole Polytechnique Federale de Lausanne, 2017-09-22) Ichim, Alexandru Eugen
Creating digital representations of humans is of utmost importance for applications ranging from entertainment (video games, movies) to human-computer interaction and even psychiatrical treat- ments. What makes building credible digital doubles difficult is the fact that the human vision system is very sensitive to perceiving the complex expressivity and potential anomalies in body structures and motion. This thesis will present several projects that tackle these problems from two different perspectives: lightweight acquisition and physics-based simulation. It starts by describing a complete pipeline that allows users to reconstruct fully rigged 3D facial avatars using video data coming from a handheld device (e.g., smartphone). The avatars use a novel two-scale representation composed of blendshapes and dynamic detail maps. They are constructed through an optimization that integrates feature tracking, optical flow, and shape from shading. Continuing along the lines of accessible acquisition systems, we discuss a framework for simultaneous tracking and modeling of articulated human bodies from RGB-D data. We show how L1 regularization can be used to extract semantic information for the body shapes. In the second half of the thesis, we will deviate from using standard linear reconstruction and animation models, and rather focus on exploiting physics-based techniques that are able to incorporate complex phenomena such as dynamics, collision response and incompressibility of the materials. The first approach we propose assumes that each 3D scan of an actor records his body in a physical steady state and uses a process called inverse physics to extract a volumetric physics-ready anatomical model of him. By using biologically-inspired growth models for the bones, muscles and fat, our method can obtain realistic anatomical reconstructions that can be later on animated using external tracking data such as the one resulting from tracking motion capture markers. This is then extended to a novel physics-based approach for facial reconstruction and animation. We propose a novel facial reconstruction and animation model which simulates biomechanical muscle contractions in a volumetric face model in order to create the facial expressions seen in the input scans. We then show how this approach allows for new avenues of dynamic artistic control, simulation of corrective facial surgery, and interaction with external forces and objects.
Operator Representations in Geometry Processing
(2017) Azencot, Omri
This thesis introduces fundamental equations as well as discrete tools and numerical methods for carrying out various geometrical tasks on three-dimensional surfaces via operators. An example for an operator is the Laplacian which maps real-valued functions to their sum of second derivatives. More generally, many mathematical objects feature an operator interpretation, and in this work, we consider a few of them in the context of geometry processing and numerical simulation problems. The operator point of view is useful in applications since high-level algorithms can be devised for the problems at hand with operators serving as the main building blocks. While this approach has received some attention in the past, it has not reached its full potential, as the following thesis tries to hint. The contribution of this document is twofold. First, it describes the analysis and discretization of derivations and related operators such as covariant derivative, Lie bracket, pushforward and flow on triangulated surfaces. These operators play a fundamental role in numerous computational science and engineering problems, and thus enriching the readily available differential tools with these novel components offers multiple new avenues to explore. Second, these objects are then used to solve certain differential equations on curved domains such as the advection equation, the Navier– Stokes equations and the thin films equations. Unlike previous work, our numerical methods are intrinsic to the surface—that is, independent of a particular geometry flattening. In addition, the suggested machinery preserves structure—namely, a central quantity to the problem, as the total mass, is exactly preserved. These two properties typically provide a good balance between computation times and quality of results. From a broader standpoint, recent years have brought an expected increase in computation power along with extraordinary advances in the theory and methodology of geometry acquisition and processing. Consequently, many approaches which were infeasible before, became viable nowadays. In this view, the operator perspective and its application to differential equations, as depicted in this work, provides an interesting alternative, among the other approaches, for working with complex problems on non-flat geometries. In the following chapters, we study in which cases operators are applicable, while providing a fair comparison to state-of-the-art methods.
High dynamic range imaging: problems of video exposure bracketing, luminance calibration and gloss editing
(2017-06-02) Gryaditskaya, Yulia
Two-dimensional, conventional images are gradually losing their hegemony, leaving room for novel formats. Among these, 8 bit images give place to high dynamic range (HDR) image formats, allowing to improve colour gamut and visibility of details in dark and bright areas leading to a more immersive viewing experience. It opens wide opportunities for post-processing, which can be useful for artistic rendering, enhancement of viewing experience or medical applications. Simultaneously, light-field scene representation as well is gaining importance, propelled by the recent reappearance of virtual reality, the improvement of both acquisition techniques, and computational and storage capabilities. Light-field data as well allows to achieve a broad range of effects in post-production: among others, it enables a change of a camera position, an aperture or a focal length. It facilitates object insertions and simplifies visual effects workflow by integrating 3D nature of visual effects with 3D nature of light fields. Content generation is one of the stumbling blocks in these realms. Sensor limitations of a conventional camera do not allow to capture wide dynamic range. This especially is the case for mobile devices, where small sensors are optimised for capturing in high-resolution. The "HDR mode" often encountered on such devices, relies on techniques called "exposure fusion" and allows to partially overcome the limited range of a sensor. The HDR video at the same time remains a challenging problem. We suggest a solution for an HDR video capturing on a mobile device. We analyse dynamic range of motion regions, the regions which are the most prone to reconstruction artefacts, and suggest a real-time exposure selection algorithm. Further, an HDR content visualization task often requires an input to be in absolute values. We address this problem by presenting a calibration algorithm that can be applied to existent imagery and does not require any additional measurement hardware. Finally, as light fields use becomes more common, a key challenge is the ability to edit or modify the appearance of the objects in the light field. To this end, we propose a multidimensional filtering approach in which the specular highlights are filtered in the spatial and angular domains to target a desired increase of the material roughness.
A Generalized Proceduralization Framework for Urban Models with Applications in Procedural Modeling, Synthesis, and Reconstruction
(ProQuest Dissertations Publishing, 2017) Demir, Ilke
The technological developments in the last century have carried us from a few pixels per screen to infinite worlds. Consecutively, recent bottleneck in graphics industry has been changing from the “means” to the “objects”, in other words, from the technology to the content. In particular, architectural models have always played an important role in areas like computer graphics, virtual environments, and urban planning. Because of the proliferation of such areas, the demand for city-scale 3D urban models has significantly increased, as well as the availability of 3D data sets of buildings and other urban structures. One option is to manually create such content, however humans are expensive and slow, and more and more are needed as the content to be created explodes for movies, games, and visualizations. Another option is to efficiently re-use the existing large set of 3D polygonal models available from public databases, created by scans, images, time-of-flight cameras, manual modeling, etc. However the results of these approaches usually lack high-level grouping or segmentation information which hampers efficient re-use and synthesis. Procedural models are known to be an effective solution to create massive amount of content from powerful and compact parameterized representations. While procedural modeling provides compelling architectural structures, creating detailed and realistic buildings needs time and extensive coding as well as significant domain expertise. In other words, procedural modeling is a solution for content creation, however lack of artistic control and the need for domain expertise make it challenging. Observing the content creation problem and the challenges of procedural modeling, we identified and developed a set of proceduralization tools, that convert existing models into an easy to manipulate procedural form and allow quick synthesis of visually similar objects. The central idea of our research is that we can automate and assist modeling and creation tasks by proceduralization of existing models such as architectural meshes, building point clouds, or textured urban areas. The geometrical and visual information hidden in already existing models actually contain high-level semantic and structural information, so a proceduralization framework can reveal such information for a variety of purposes. Our proceduralization framework for urban spaces allows the re-use of existing models in various formats by first conducting shape analysis methods on such models to find and exploit the repetitions and similarities revealing the hidden high-level structural information, and then using grammar discovery methods on those components to extract a representative grammar (or a procedural form) of the model. We have shown that the structural information and repetitions inside already existing models can be exploited to obtain the high-level representation hidden in such models for making the design and modeling of urban content more efficient and intuitive. Initial applications of our work enables the benefits of procedural modeling on existing models such as compression, rendering, and content generation. Additionally, we have proposed new applications of such procedural representations, including reconstruction of incomplete models, geo-localization within urban areas, and structure preserving synthesis systems. We expect our effort will enlighten the artists' and designers' creation process by converting the existing modeling tools into faster, easier to implement, more interactive, and more intuitive procedural modeling systems, using the power of proceduralization.
Controllable Shape Synthesis for Digital Fabrication
(Université de Lorraine, 2017-02-03) Dumas, Jérémie
The main goal of this thesis is to propose methods to synthesize shapes in a controllable manner, with the purpose of being fabricated. As 3D printers grow more accessible than ever, modeling software must now take into account fabrication constraints posed by additive manufacturing technologies. Consequently, efficient algorithms need to be devised to model the complex shapes that can be created through 3D printing. We develop algorithms for by-example shape synthesis that consider the physical behavior of the structure to fabricate. All the contributions of this thesis focus on the problem of generating complex shapes that follow geometric constraints and structural objectives. In a first time, we focus on dealing with fabrication constraints, and propose a method for synthesizing efficient support structures that are well-suited for filament printers. In a second time, we take into account appearance control, and develop new by-example synthesis methods that mixes in a meaningful manner criteria on the appearance of the synthesized shapes, and constraints on their mechanical behavior. Finally, we present a highly scalable method to control the elastic properties of printed structures. We draw inspiration from procedural texture synthesis methods, and propose an efficient algorithm to synthesize printable microstructures with controlled elastic properties
Efficient Illumination Algorithms for Global Illumination in Interactive and Real-Time Rendering
(National Archive of PhD Theses, 2017-11-22) Vardis, Konstantinos
The synthesis of photorealistic digital imagery has long been considered as one of the most fascinating domains in the field of computer graphics. Its main goal is the generation of visually stunning images, mimicking as close as possible the appearance of objects in the physical world. The endeavor for visual realism has directed a large amount of research interest in the investigation of the interactions of light and matter, resulting in an established mathematical framework and the striking beauty of the generated images on today’s production level renderers. While the theoretical concepts of light transport are well understood and applied in offline rendering, the interactive reproduction of the underlying physical processes remains a highly challenging topic due to the various constraints involved in the process. Furthermore, the increased need for the delivery of highly dynamic interactive content in today’s vast virtual environments, that can potentially change in every frame, has undoubtedly increased the necessity for highly efficient, interactive illumination algorithms. In this thesis, we investigate such methods, in the field of photorealistic image synthesis. Our contributions focus exclusively on the development of robust algorithms for real-time and interactive global illumination, under the considerations of fully dynamic content. Regarding real-time rendering, the majority of algorithms are based on the rasterization pipeline, where the support of dynamic content is inherently provided. However, the strict time restrictions of real-time applications pose significant constraints in the operation of the computationally demanding global illumination algorithms, severely impacting the resulting quality of the rendered images. There, the most common approximation is imposed on the algorithmic input where, typically, the highly-detailed geometric information is replaced by either (i) a partial (layered), view-dependent and discretized representation, or a (ii) view-independent but, crude, regular subdivision of the environment in image- and volume-space methods, respectively. We contribute to the domain of real-time rendering by proposing two novel techniques, focusing particularly on the improvement of the visual instability of prior approaches as well as on the efficiency of the underlying representations. First, we propose a generic method to efficiently address the view-dependent inconsistencies of image-domain methods, demonstrated on screen-space ambient occlusion. This is accomplished by taking advantage of buffers containing geometric information from other view points, already generated as part of the rendering process, such as the shadow maps. Second, we improve the efficiency as well as on the visual stability of volume-based global illumination methods, by introducing the idea of directional chrominance subsampling for radiance field compression, an optimized cache point population scheme and a view-independent approximate indirect shadowing technique. In order to support dynamic content in interactive applications, the effort of the research community has been heavily focused on the improvement of the efficiency of the ray tracing pipeline, which has been traditionally employed for production rendering. However, the computational overhead of the required complex acceleration structures is still restricting these approaches to partially static content. Alternatively, a recent category of techniques have attempted to exploit the rasterization pipeline, which inherently supports dynamic environments, to achieve quality identical to the ray tracing pipeline. Still, the proposed solutions are not yet able to support a full global illumination algorithm without posing any restrictions on the geometric representation or on the effects that can be reproduced. In the domain of interactive rendering, we present two methods that investigate the ability of the modern rasterization pipeline to provide a viable alternative to the costly construction stages of spatial acceleration structures. The proposed methods are able to perform high-quality interactive ray tracing in arbitrarily complex and dynamic environments, thus lifting the limitations of prior rasterization-based methods. Our first method ii employs multifragment rendering techniques to effectively capture, for the first time, a highly-detailed representation of the entire environment where a full global illumination algorithm, such as path tracing, can be elegantly supported. Ray tracing is efficiently achieved by exploiting image-space empty space skipping and approximate ray-fragment intersection tests. The presented solution advances the field of image-space ray tracing and provides small construction times as well as scalable traversal performance. However, the resulting quality is approximate and can suffer from high memory overhead due to its fragment-based data structure. Our second approach, which completes our investigation, applies the deferred nature of the traditional ray-tracing pipeline in a rasterization-based framework. Thus, we are able to exploit a primitive-based acceleration data structure and support three, conflicting in prior work, objectives: (i) dynamic environments through fast construction times, (ii) quality identical to the ray tracing pipeline via primitive-based intersection tests, and (iii) reduced memory requirements. Additionally, the presented method further generalizes on the field of image-space ray tracing by exploiting various empty space skipping optimization strategies in order to efficiently support accurate ray-primitive intersection queries.
Enhancing Digital Fabrication with Advanced Modeling Techniques
(2017-04) Malomo, Luigi
A few years ago there were only expensive machineries dedicated to rapid prototyping for professionals or industrial application, while nowadays very affordable solutions are on the market and have become useful tools for experimenting, providing access to final users. Given the digital nature of these machine-controlled manufacturing processes, a clear need exists for computational tools that support this new way of productional thinking. For this reason the ultimate target of this research is to improve the easiness of use of such technologies, providing novel supporting tools and methods to ultimately sustain the concept of democratized design (“fabrication for the masses”). In this thesis we present a novel set of methods to enable, with the available manufacturing devices, new cost-effective and powerful ways of producing objects. The contributions of the thesis are three. The first one is a technique that allows to automatically create a tangible illustrative representation of a 3D model by interlocking together a set of planar pieces, which can be fabricated using a 2D laser cutter. The second method allows to automatically design flexible reusable molds to produce many copies of an input digital object. The designs produced by this method can be directly sent to a 3D printer and used to liquid-cast multiple replicas using a wide variety of materials. The last technique is a method to fabricate, using a single-material 3D printer, objects with custom elasticity, and an optimization strategy that, varying the elastic properties inside the volume, is able to design printable objects with a prescribed mechanical behavior.
Adjoint-Driven Importance Sampling in Light Transport Simulation
(Charles University, Prague, 2017-06-26) Vorba, Jiří
Monte Carlo light transport simulation has recently been adopted by the movie industry as a standard tool for producing photo realistic imagery. As the industry pushes current technologies to the very edge of their possibilities, the unprecedented complexity of rendered scenes has underlined a fundamental weakness of MC light transport simulation: slow convergence in the presence of indirect illumination. The culprit of this poor behaviour is that the sam- pling schemes used in the state-of-the-art MC transport algorithms usually do not adapt to the conditions of rendered scenes. We base our work on the ob- servation that the vast amount of samples needed by these algorithms forms an abundant source of information that can be used to derive superior sampling strategies, tailored for a given scene. In the ﬁrst part of this thesis, we adapt general machine learning techniques to train directional distributions for biasing scattering directions of camera paths towards incident illumination (radiance). Our approach allows progressive training from a stream of particles while main- taining bounded memory footprint. This progressive nature makes the method robust even in scenarios where we have little information in the early stages of the training due to diﬃcult visibility. The proposed method is not restricted only to path tracing, where paths start at the camera, but can be employed also in light tracing or photon mapping, where paths are emitted from light sources, as well as in combined bidirectional methods. In the second part of this thesis we revisit Russian roulette and splitting, two vari- ance reduction techniques that have been used in computer graphics for more than 25 years. So far, however, the path termination (Russian roulette) and splitting rates have been based only on local material properties in the scene which can re- sult in ineﬃcient simulation in the presence of indirect illumination. In contrast, we base the termination and splitting rates on a pre-computed approximation of the adjoint quantity (i.e. radiance in the case of path tracing) which yields superior results to previous approaches. To increase robustness of our method, we adopt the so called weight window, a standard technique in neutron transport simulations. Both methods, that is the biasing of scattering directions introduced in the ﬁrst part of the thesis and the adjoint-driven Russian roulette and splitting, are based on the prior estimate of the adjoint quantity. Nevertheless, they consti- tute two complementary importance sampling strategies of transported light and as we show, their combination yields superior results to each strategy alone. As one of our contributions, we present a theoretical analysis that provides insights into the importance sampling properties of our adjoint-driven Russian roulette and splitting, and also explains the synergic behaviour of the two strategies.
Face2Face: Real-time Facial Reenactment
(2017) Thies, Justus
In this dissertation we show our advances in the field of 3D reconstruction of human faces using commodity hardware. Beside the reconstruction of the facial geometry and texture, real-time face tracking is demonstrated. The developed algorithms are based on the principle of analysis-by-synthesis. To apply this principle, a mathematical model that represents a face virtually is defined. In addition to the face, the sensor observation process of the used camera is modeled. Utilizing this model to synthesize facial imagery, the model parameters are adjusted, such that the synthesized image fits the input image as good as possible. Thus, in reverse, this process transfers the input image to a virtual representation of the face. The achieved quality allows many new applications that require a good reconstruction of the face. One of these applications is the so-called ''Facial Reenactment''. Our developed methods show that such an application does not need any special hardware. The generated results are nearly photo-realistic videos that show the transfer of the mimic of one person to another person. These techniques can for example be used to bring movie dubbing to a new level. Instead of adapting the audio to the video, which might also include changes of the text, the video can be post-processed to match the mouth movements of the dubber. Since the approaches that we show in this dissertation run in real-time, one can also think of a live dubber in a video teleconferencing system that simultaneously translates the speech of a person to another language. The published videos of our projects in this dissertation led to a broad discussion in the media. On the one hand this is due to the fact that our methods are designed such that they run in real-time and on the other hand that we reduced the hardware requirements to a minimum. In fact, after some preprocessing, we are able to edit ordinary videos from the Internet in real-time. Amongst others, we impose a different mimic to faces of prominent persons like former presidents of the United States of America. This led inevitably to a discussion about trustworthiness of video material, especially from unknown source. Most people did not expect that such manipulations are possible, neglecting existing methods that are already able to edit videos (e.g. special effects in movie productions). Thus, beside the advances in real-time face tracking, our projects raised the awareness of video manipulation.
Moment-Based Methods for Real-Time Shadows and Fast Transient Imaging
(Universitäts- und Landesbibliothek Bonn, 2017-12-06) Peters, Christoph
We apply the theory of moments to develop computationally efficient methods for real-time rendering of shadows and reconstruction of transient images from few measurements. Given moments of an unknown probability distribution, i.e. the expectations of known, real random variables, the theory of moments strives to characterize all distributions that could have led to these moments. Earlier works in computer graphics only use the most basic results of this powerful theory. When filtering shadows based on shadow maps, the distribution of depth values within the filter region has to be estimated. Variance shadow mapping does this using two power moments. While this linear representation admits direct filtering, it leads to a very coarse reconstruction. We generalize this approach to use an arbitrary set of general moments and benchmark thousands of possible choices. Based on the results, we propose the use of moment shadow mapping which produces high-quality antialiased shadows efficiently by storing four power moments in 64 bits per shadow map texel. Techniques for shadow map filtering have been applied to a variety of problems. We combine these existing approaches with moment shadow mapping to render shadows of translucent occluders using alpha blending, soft shadows using summed-area tables and prefiltered single scattering using six power moments. All these techniques have a high overhead per texel of the moment shadow map but a low overhead per shaded pixel. Thus, they scale well to the increasingly high resolutions of modern displays. Transient images help to analyze light transport in scenes. Besides two spatial dimensions, they are resolved in time of flight. Earlier cost-efficient approaches reconstruct them from measurements of amplitude modulated continuous wave lidar systems but they typically take more than a minute of capture time. We pose this reconstruction problem as trigonometric moment problem. The maximum entropy spectral estimate and the Pisarenko estimate are known closed-form solutions to such problems which yield continuous and sparse reconstructions, respectively. By applying them, we reconstruct complex impulse responses with m distinct returns from measurements at as few as m non-zero frequencies. For m=3 our experiments with measured data confirm this. Thus, our techniques are computationally efficient and simultaneously reduce capture times drastically. We successfully capture 18.6 transient images per second which leads to transient video. As an important byproduct, this fast and accurate reconstruction of impulse responses enables removal of multipath interference in range images.
Geometric Deep Learning for Shape Analysis
(Università della Svizzera italiana, 2017-08-16) Boscaini, Davide
The past decade in computer vision research has witnessed the re-emergence of artificial neural networks (ANN), and in particular convolutional neural net- work (CNN) techniques, allowing to learn powerful feature representations from large collections of data. Nowadays these techniques are better known under the umbrella term deep learning and have achieved a breakthrough in perfor- mance in a wide range of image analysis applications such as image classification, segmentation, and annotation. Nevertheless, when attempting to apply deep learning paradigms to 3D shapes one has to face fundamental differences between images and geometric objects. The main difference between images and 3D shapes is the non-Euclidean nature of the latter. This implies that basic operations, such as linear combination or convolution, that are taken for granted in the Euclidean case, are not even well defined on non-Euclidean domains. This happens to be the major obstacle that so far has precluded the successful application of deep learning methods on non-Euclidean geometric data. The goal of this thesis is to overcome this obstacle by extending deep learning tecniques (including, but not limiting to CNNs) to non-Euclidean domains. We present different approaches providing such extension and test their effectiveness in the context of shape similarity and correspondence applications. The proposed approaches are evaluated on several challenging experiments, achieving state-of- the-art results significantly outperforming other methods. To the best of our knowledge, this thesis presents different original contributions. First, this work pioneers the generalization of CNNs to discrete manifolds. Second, it provides an alternative formulation of the spectral convolution operation in terms of the windowed Fourier transform to overcome the drawbacks of the Fourier one. Third, it introduces a spatial domain formulation of convolution operation using patch operators and several ways of their construction (geodesic, anisotropic diffusion, mixture of Gaussians). Fourth, at the moment of publication the proposed approaches achieved state-of-the-art results in different computer graphics and vision applications such as shape descriptors and correspondence.
Dynamic and Probabilistic Point-Cloud Processing
(2017) Preiner, Reinhold Dr.
In recent years, the scanning of real-world physical objects using 3d acquisition devices has become an everyday task. Such 3d scanners output a set of three-dimensional points, called point cloud, sampling the surface of an object. These samples are typically subject to imperfections like noise, holes and outliers. In computer graphics, the field of surface reconstruction is concerned with the problem of converting this raw point data to a clean, more usable and visualizable representation. Some of the developed techniques aim at directly rendering a closed surface from such a point cloud, but also require certain precomputations to produce images of high quality. The ever-increasing acquisition rates and data throughputs of even low-cost scanner hardware have recently made it possible to capture dynamic and animated objects in real time. These high-speed acquisition capabilities call for new high-performance reconstruction algorithms that are able to keep up with these acquisition rates and allow an instant high-quality visualization of the real-time captured data. In this thesis, we pick up on this problem and develop various techniques that allow a fast processing and visualization of such raw, unstructured and potentially dynamic point clouds, which might be streamed to our computer at real-time rates from any possible source. In the course of our work, we investigate probabilistic methods that allow achieving a significant acceleration of state-of-the-art point-based operators, and use statistical models of 3d point sets to develop a fast technique for probabilistic surface reconstruction and representation. We develop a GPU point-rendering framework that performs any reconstruction computations required for a high-quality visualization instantly, i.e., on the fly at render time, and only on a necessary minimal subset of the data, i.e., the points visible on the screen. To this end, the first part of this thesis addresses a basic problem common to almost any surface-reconstruction technique, which is the fast and efficient search for spatial neighbors in an unstructured and unordered large collection of points. Knowing about a point’s nearest neighbors is an essential prerequisite for establishing local connectivity, assessing the shape of the surrounding surface, and applying filter operations for improving the quality of the geometric data and thus the resulting surface. In the second part, we improve on this direct reconstruction and rendering technique and present a more elaborate method that allows working at arbitrary reconstruction bandwidths, improves on the temporal stability of the rendered image, and produces a surface rendering of increased smoothness. In the third part, we focus on the problem of noise and outliers in the input data, and introduce a novel technique that allows for a fast feature-preserving resampling of unstructured dynamic point sets at render time. To this end, we describe the point cloud by a sparse probabilistic Gaussian mixture model, which allows for a much more compact representation and thus much faster operations on the spatial data. We will show that this technique significantly improves on the speed and even on the accuracy and quality of a feature-preserving point-set resampling operator. Based on the observed computational benefits of this probabilistic model, the final part of this thesis investigates a new way of defining a smooth and continuous surface solely based on a sparse Gaussian mixture. We will develop an entirely probabilistic reconstruction pipeline, and show that we can describe a feature-rich surface in a highly memory-efficient way while obtaining a reconstruction performance that can compete and even improve on the state of the art.
Reducing Animator Keyframes
(2017) Holden, Daniel
The aim of this doctoral thesis is to present a body of work aimed at reducing the time spent by animators manually constructing keyframed animation. To this end we present a number of state of the art machine learning techniques applied to the domain of character animation. Data-driven tools for the synthesis and production of character animation have a good track record of success. In particular, they have been adopted thoroughly in the games industry as they allow designers as well as animators to simply specify the high-level descriptions of the animations to be created, and the rest is produced automatically. Even so, these techniques have not been thoroughly adopted in the film industry in the production of keyframe based animation. Due to this, the cost of producing high quality keyframed animation remains very high, and the time of professional animators is increasingly precious. We present our work in four main chapters. We first tackle the key problem in the adoption of data-driven tools for key framed animation - a problem called the inversion of the rig function. Secondly, we show the construction of a new tool for data-driven character animation called the motion manifold - a representation of motion constructed using deep learning that has a number of properties useful for animation research. Thirdly, we show how the motion manifold can be extended as a general tool for performing data-driven animation synthesis and editing. Finally, we show how these techniques developed for keyframed animation can also be adapted to advance the state of the art in the games industry.
Topological Changes in Simulations of Deformable Objects
(Université de Strasbourg and archives-ouvertes.fr, 2017-04) Paulus, Christoph J
Virtual cutting of deformable objects is at the core of many applications in interactive simulation and especially in computational medicine. The ability to simulate surgical cuts, soft tissue tearing or fractures, is essential for aug- menting the capabilities of existing or future simulation systems. In this thesis, we present a new remeshing algorithm based on the finite element method. For tetrahedral elements with linear shape functions, we combined remeshing algorithm with the movement of the nodes to the cutting surface, called snapping in the literature. Our approach shows benefits when evaluat- ing the impact of cuts on the number of nodes and the numerical quality of the mesh. When applying our remeshing algorithm to quadratic shape functions, we observe similar results. Due to the curved surfaces of the elements, when using quadratic shape functions, the snapping of nodes entails higher chal- lenges. Thus, to investigate into the snapping, experience has been gathered on triangular shell elements, simulating fractures. Beyond the simulation of fractures, our remeshing approach for tetrahedral elements is generic enough to support a large variety of applications. In this work, we are the first to present results on the detection of topological changes, such as fractures, tearing and cutting, from a monocular video stream. Ex- amples with highly elastic silicone bands, in-vivo livers and ex-vivo kidneys show the robustness of our detection algorithm, which is combined with the remeshing approach, in different scenarios. Finally, the augmentation of in- ternal organ structures highlights the clinical potential and importance of the conducted work.
Realistic Visualization of Accessories within Interactive Simulation Systems for Garment Prototyping
(2017-03-03) Knuth, Martin
In virtual garment prototyping, designers create a garment design by using Computer Aided Design (CAD). In difference to traditional CAD the word "aided" in this case refers to the computer replicating real world behavior of garments. This allows the designer to interact naturally with his design. The designer has a wide range of expressions within his work. This is done by defining details on a garment which are not limited to the type of cloth used. The way how cloth patterns are sewn together and the style and usage of details of the cloth’s surface, like appliqués, have a strong impact on the visual appearance of a garment to a large degree. Therefore, virtual and real garments usually have a lot of such surface details. Interactive virtual garment prototyping itself is an interdisciplinary field. Several prob- lems have to be solved to create an efficiently usable real-time virtual prototyping system for garment manufacturers. Such a system can be roughly separated into three sub-components. The first component deals with acquisition of material and other data needed to let a sim- ulation mimic plausible real world behavior of the garment. The second component is the garment simulation process itself. Finally, the third component is centered on the visualiza- tion of the simulation results. Therefore, the overall process spans several scientific areas which have to take into account the needs of each other in order to get an overall interactive system. In my work I especially target the third section, which deals with the visualization. On the scientific side, the developments in the last years have shown great improvements on both speed and reliability of simulation and rendering approaches suitable for the virtual prototyping of garments. However, with the currently existing approaches there are still many problems to be solved, especially if interactive simulation and visualization need to work together and many object and surface details come into play. This is the case when using a virtual prototyping in a productive environment. The currently available approaches try to handle most of the surface details as part of the simulation. This generates a lot of data early in the pipeline which needs to be transferred and processed, requiring a lot of processing time and easily stalls the pipeline defined by the simulation and visualization system. Additionally, real world garment examples are already complicated in their cloth arrangement alone. This requires additional computational power. Therefore, the interactive garment simulation tends to lose its capability to allow interactive handling of the garment. In my work I present a solution, which solves this problem by moving the handling of design details from the simulation stage entirely to a completely GPU based rendering stage. This way, the behavior of the garment and its visual appearance are separated. There- fore, the simulation part can fully concentrate on simulating the fabric behavior, while the visualization handles the placing of surface details lighting, materials and self-shadowing. Thus, a much higher degree of surface complexity can be achieved within an interactive virtual prototyping system as can be done with the current existing approaches.
Smarter screen space shading
(2017-11-10) Nalbach, Oliver
This dissertation introduces a range of new methods to produce images of virtual scenes in a matter of milliseconds. Imposing as few constraints as possible on the set of scenes that can be handled, e.g., regarding geometric changes over time or lighting conditions, precludes pre-computations and makes this a particularly difficult problem. We first present a general approach, called deep screen space, using which a variety of light transport aspects can be simulated within the aforementioned setting. This approach is then further extended to additionally handle scenes containing participating media like clouds. We also show how to improve the correctness of deep screen space and related algorithms by accounting for mutual visibility of points in a scene. After that, we take a completely different point of view on image generation using a learning-based approach to approximate a rendering function. We show that neural networks can hallucinate shading effects which otherwise have to be computed using costly analytic computations. Finally, we contribute a holistic framework to deal with phosphorescent materials in computer graphics, covering all aspects from acquisition of real materials, to easy editing, to image synthesis.
A Microservice Architecture for the Processing of Large Geospatial Data in the Cloud
(2017-11-07) Krämer, Michel
With the growing number of devices that can collect spatiotemporal information, as well as the improving quality of sensors, the geospatial data volume increases con- stantly. Before the raw collected data can be used, it has to be processed. Current- ly, expert users are still relying on desktop-based Geographic Information Systems to perform processing workflows. However, the volume of geospatial data and the complexity of processing algorithms exceeds the capacities of their workstations. There is a paradigm shift from desktop solutions towards the Cloud, which offers virtually unlimited storage space and computational power, but developers of pro- cessing algorithms often have no background in computer science and hence no expertise in Cloud Computing. Our research hypothesis is that a microservice architecture and Domain-Specific Languages can be used to orchestrate existing geospatial processing algorithms, and to compose and execute geospatial workflows in a Cloud environment for efficient application development and enhanced stakeholder experience. We present a soft- ware architecture that contains extension points for processing algorithms (or mi- croservices), a workflow management component for distributed service orchestra- tion, and a workflow editor based on a Domain-Specific Language. The main aim is to provide both users and developers with the means to leverage the possibilities of the Cloud, without requiring them to have a deep knowledge of distributed com- puting. In order to conduct our research, we follow the Design Science Research Methodology. We perform an analysis of the problem domain and collect require- ments as well as quality attributes for our architecture. To meet our research objec- tives, we design the architecture and develop approaches to workflow management and workflow modelling. We demonstrate the utility of our solution by applying it to two real-world use cases and evaluate the quality of our architecture based on defined scenarios. Finally, we critically discuss our results. Our contributions to the scientific community can be classified into three pillars. We present a scalable and modifiable microservice architecture for geospatial pro- cessing that supports distributed development and has a high availability. Further, we present novel approaches to service integration and orchestration in the Cloud as well as rule-based and dynamic workflow management without a priori design-time knowledge. For the workflow modelling we create a Domain-Specific Language that is based on a novel language design method. Our evaluation results support our hypothesis. The microservice architectural style enables efficient development of a distributed system. The Domain-Specific Language and our approach to service integration enhance stakeholder experience. Our work is a major step within the paradigm shift towards the Cloud and opens up possibilities for future research.
The Impact of Virtual Embodiment on Perception, Attitudes, and Behaviour
(TDX (Tesis Doctorals en Xarxa), 2017-04-26) Banakou, Domna
Over the past two decades extensive research in experimental psychology, cognitive neuroscience, and virtual reality has provided evidence for the malleability of our brain's body representation. It has been shown that a person's body can be substituted by a life-sized artificial one, resulting in a perceptual illusion of body ownership over the fake body. Interestingly, several studies have shown that when people are virtually represented with a body different to their own, they exhibit behaviours associated with attributes pertaining to that body. In the research described here we exploited Immersive Virtual Reality to induce body ownership illusions over distinct virtual bodies. We examined how an altered self-representation can influence one's self-perception, perception of the environment, and implicit biases. To this end, we carried out two experimental studies to investigate embodiment of adults in a child virtual body, and a different race virtual body. Moreover, we tested whether it is possible to induce illusory agency over specific actions that are not carried out by the participants themselves. In the Virtual Child Body study, we embodied adults both as a 4-year-old child, and as an adult scaled-down to the same height as the child. The results showed that embodiment in the child body led to a significant overestimation of object sizes, which was approximately double the overestimation of those embodied in the adult body. Moreover, embodiment in the child resulted in changes in implicit attitudes about the self towards being child-like. These findings were diminished under asynchronous visuomotor correlations, providing further proof for the importance of visuomotor contingencies in producing body ownership illusions. Our findings extend and enrich previous research, yielding additional evidence of the malleability of our body representation. In the Racial Bias study, we aimed to explore how the type of body can influence racial discrimination, by embodying white people in a black virtual body. Previous research has shown that this type of embodiment can lead to a reduction of implicit racial bias, but its long-term effects were unknown. Here we tested whether this reduction in implicit bias can (a) be replicated, (b) it can last for at least one week, and (c) it is enhanced by multiple exposures. Participants were immersed in a virtual scenario between one and three times, each separated by two days, and implicit bias was measured one week before their first exposure, and one week after their last. The results showed that implicit bias decreased more for those with the black virtual body than the white, even a week after their virtual exposure, and irrespective of the number of exposures. In the Illusory Speaking study, we explored the possibility of inducing illusory agency over an action that participants did not carry out themselves. We describe a set of experiments, where under appropriate sensorimotor contingencies, we induce a subjective illusion of agency over the participants' speaking virtual body, as if they had been themselves speaking. When participants were asked to speak after this exposure, they shifted the fundamental frequency of their utterances towards that of the stimulus voice of the virtual body. We argue that these findings can be reconciled with current theories of agency, provided that the critical role of both ownership and actual agency over the virtual body are taken into account. Overall, our studies expand previous evidence for the malleability of our body representation, demonstrating how it is possible to induce ownership illusions over a child body, a different race body, or even a speaking body. Notably, we provide evidence of how body ownership and agency over the virtual body result in powerful, lasting changes in perceptual and cognitive processing, having the potential of compelling applications in psychology and neuroscience.

Browse

Recent Submissions

Results Per Page

Sort Options