40-Issue 1

Permanent URI for this collection

https://diglib.eg.org/handle/10.2312/2633015

Browse

Now showing 1 - 20 of 33

Adaptive Compositing and Navigation of Variable Resolution Images
(© 2021 Eurographics ‐ The European Association for Computer Graphics and John Wiley & Sons Ltd, 2021) Licorish, C.; Faraj, N.; Summa, B.; Benes, Bedrich and Hauser, Helwig
We present a new, high‐quality compositing pipeline and navigation approach for variable resolution imagery. The motivation of this work is to explore the use of variable resolution images as a quick and accessible alternative to traditional gigapixel mosaics. Instead of the common tedious acquisition of many images using specialized hardware, variable resolution images can achieve similarly deep zooms as large mosaics, but with only a handful of images. For this approach to be a viable alternative, the state‐of‐the‐art in variable resolution compositing needs to be improved to match the high‐quality approaches commonly used in mosaic compositing. To this end, we provide a novel, variable resolution mosaic seam calculation and gradient domain color correction. This approach includes a new priority order graph cuts computation along with a practical data structure to keep memory overhead low. In addition, navigating variable resolution images is challenging, especially at the zoom factors targeted in this work. To address this challenge, we introduce a new image interaction for variable resolution imagery: a pan that automatically, and smoothly, hugs available resolution. Finally, we provide several real‐world examples of our approach producing high‐quality variable resolution mosaics with deep zooms typically associated with gigapixel photography.
Anisotropic Spectral Manifold Wavelet Descriptor
(© 2021 Eurographics ‐ The European Association for Computer Graphics and John Wiley & Sons Ltd, 2021) Li, Qinsong; Hu, Ling; Liu, Shengjun; Yang, Dangfu; Liu, Xinru; Benes, Bedrich and Hauser, Helwig
In this paper, we present a powerful spectral shape descriptor for shape analysis, named Anisotropic Spectral Manifold Wavelet Descriptor (ASMWD). We proposed a novel manifold harmonic signal processing tool termed Anisotropic Spectral Manifold Wavelet Transform (ASMWT) first. ASMWT allows to comprehensively analyse signals from multiple wavelet diffusion directions on local manifold regions of the shape with a series of low‐pass and band‐pass frequency filters in each direction. Based on the ASMWT coefficients of a very simple signal, the ASMWD is efficiently constructed as a localizable and discriminative multi‐scale point descriptor. Since the wavelets used in our descriptor are direction‐sensitive and able to robustly reconstruct the signals with a finite number of scales, it makes our descriptor compact, efficient, and unambiguous under intrinsic symmetry. The extensive experiments demonstrate that our descriptor achieves significantly better performance than the state‐of‐the‐art descriptors and can greatly improve the performance of shape matching methods including both handcrafted and learning‐based methods.
Thin Cloud Removal for Single RGB Aerial Image
(© 2021 Eurographics ‐ The European Association for Computer Graphics and John Wiley & Sons Ltd, 2021) Song, Chengfang; Xiao, Chunxia; Zhang, Yeting; Sui, Haigang; Benes, Bedrich and Hauser, Helwig
Acquired above variable clouds, aerial images contain the components of ground reflection and cloud effect. Due to the non‐uniformity, clouds in aerial images are even harder to remove than haze in terrestrial images. This paper proposes a divide‐and‐conquer scheme to remove the thin translucent clouds in a single RGB aerial image. Based on colour attenuation prior, we design a kind of veiling metric that indicates the local concentration of clouds effectively. By this metric, an aerial image containing thickness‐varied clouds is segmented into multiple regions. Each region is veiled by clouds of nearly‐equal concentration, and hence subject to common assumptions, such as boundary constraint on transmission. The atmospheric light in each region is estimated by the modified local colour‐line model and composed into a spatially‐varying airlight map for the entire image. Then scene transmission is estimated and further refined by a weighted ‐norm based contextual regularization. Finally, we recover ground reflection via the atmospheric scattering model. We verify our cloud removal method on a number of aerial images containing thin clouds and compare our results with classical single‐image dehazing methods and the state‐of‐the‐art learning‐based declouding method, respectively.
Primitive Object Grasping for Finger Motion Synthesis
(© 2021 Eurographics ‐ The European Association for Computer Graphics and John Wiley & Sons Ltd, 2021) Hwang, Jae‐Pyung; Park, Gangrae; Suh, Il Hong; Kwon, Taesoo; Benes, Bedrich and Hauser, Helwig
We developed a new framework to generate hand and finger grasping motions. The proposed framework provides online adaptation to the position and orientation of objects and can generate grasping motions even when the object shape differs from that used during motion capture. This is achieved by using a mesh model, which we call primitive object grasping (POG), to represent the object grasping motion. The POG model uses a mesh deformation algorithm that keeps the original shape of the mesh while adapting to varying constraints. These characteristics are beneficial for finger grasping motion synthesis that satisfies constraints for mimicking the motion capture sequence and the grasping points reflecting the shape of the object. We verify the adaptability of the proposed motion synthesizer according to its position/orientation and shape variations of different objects by using motion capture sequences for grasping primitive objects, namely, a sphere, a cylinder, and a box. In addition, a different grasp strategy called a three‐finger grasp is synthesized to validate the generality of the POG‐based synthesis framework.
Automatic Image Checkpoint Selection for Guider‐Follower Pedestrian Navigation
(© 2021 Eurographics ‐ The European Association for Computer Graphics and John Wiley & Sons Ltd, 2021) Kwan, K. C.; Fu, H.; Benes, Bedrich and Hauser, Helwig
In recent years guider‐follower approaches show a promising solution to the challenging problem of last‐mile or indoor pedestrian navigation without micro‐maps or indoor floor plans for path planning. However, the success of such guider‐follower approaches is highly dependent on a set of manually and carefully chosen image or video checkpoints. This selection process is tedious and error‐prone. To address this issue, we first conduct a pilot study to understand how users as guiders select critical checkpoints from a video recorded while walking along a route, leading to a set of criteria for automatic checkpoint selection. By using these criteria, including visibility, stairs and clearness, we then implement this automation process. The key behind our technique is a lightweight, effective algorithm using left‐hand‐side and right‐hand‐side objects for path occlusion detection, which benefits both automatic checkpoint selection and occlusion‐aware path annotation on selected image checkpoints. Our experimental results show that our automatic checkpoint selection method works well in different navigation scenarios. The quality of automatically selected checkpoints is comparable to that of manually selected ones and higher than that of checkpoints by alternative automatic methods.
Interactive Optimization of Generative Image Modelling using Sequential Subspace Search and Content‐based Guidance
(© 2021 Eurographics ‐ The European Association for Computer Graphics and John Wiley & Sons Ltd, 2021) Chong, Toby; Shen, I‐Chao; Sato, Issei; Igarashi, Takeo; Benes, Bedrich and Hauser, Helwig
Generative image modeling techniques such as GAN demonstrate highly convincing image generation result. However, user interaction is often necessary to obtain desired results. Existing attempts add interactivity but require either tailored architectures or extra data. We present a human‐in‐the‐optimization method that allows users to directly explore and search the latent vector space of generative image modelling. Our system provides multiple candidates by sampling the latent vector space, and the user selects the best blending weights within the subspace using multiple sliders. In addition, the user can express their intention through image editing tools. The system samples latent vectors based on inputs and presents new candidates to the user iteratively. An advantage of our formulation is that one can apply our method to arbitrary pre‐trained model without developing specialized architecture or data. We demonstrate our method with various generative image modelling applications, and show superior performance in a comparative user study with prior art iGAN [ZKSE16].
Path‐based Monte Carlo Denoising Using a Three‐Scale Neural Network
(© 2021 Eurographics ‐ The European Association for Computer Graphics and John Wiley & Sons Ltd, 2021) Lin, Weiheng; Wang, Beibei; Yang, Jian; Wang, Lu; Yan, Ling‐Qi; Benes, Bedrich and Hauser, Helwig
Monte Carlo rendering is widely used in the movie industry. Since it is costly to produce noise‐free results directly, Monte Carlo denoising is often applied as a post‐process. Recently, deep learning methods have been successfully leveraged in Monte Carlo denoising. They are able to produce high quality denoised results, even with very low sample rate, e.g. 4 spp (sample per pixel). However, for difficult scene configurations, some details could be blurred in the denoised results. In this paper, we aim at preserving more details from inputs rendered with low spp. We propose a novel denoising pipeline that handles three‐scale features ‐ pixel, sample and path ‐ to preserve sharp details, uses an improved Res2Net feature extractor to reduce the network parameters and a smooth feature attention mechanism to remove low‐frequency splotches. As a result, our method achieves higher denoising quality and preserves better details than the previous methods.
Learning Part Generation and Assembly for Sketching Man‐Made Objects
(© 2021 Eurographics ‐ The European Association for Computer Graphics and John Wiley & Sons Ltd, 2021) Du, Dong; Zhu, Heming; Nie, Yinyu; Han, Xiaoguang; Cui, Shuguang; Yu, Yizhou; Liu, Ligang; Benes, Bedrich and Hauser, Helwig
Modeling 3D objects on existing software usually requires a heavy amount of interactions, especially for users who lack basic knowledge of 3D geometry. Sketch‐based modeling is a solution to ease the modelling procedure and thus has been researched for decades. However, modelling a man‐made shape with complex structures remains challenging. Existing methods adopt advanced deep learning techniques to map holistic sketches to 3D shapes. They are still bottlenecked to deal with complicated topologies. In this paper, we decouple the task of sketch2shape into a part generation module and a part assembling module, where deep learning methods are leveraged for the implementation of both modules. By changing the focus from holistic shapes to individual parts, it eases the learning process of the shape generator and guarantees high‐quality outputs. With the learned automated part assembler, users only need a little manual tuning to obtain a desired layout. Extensive experiments and user studies demonstrate the usefulness of our proposed system.
EMU: Efficient Muscle Simulation in Deformation Space
(© 2021 Eurographics ‐ The European Association for Computer Graphics and John Wiley & Sons Ltd, 2021) Modi, V.; Fulton, L.; Jacobson, A.; Sueda, S.; Levin, D.I.W.; Benes, Bedrich and Hauser, Helwig
EMU is an efficient and scalable model to simulate bulk musculoskeletal motion with heterogenous materials. First, EMU requires no model reductions, or geometric coarsening, thereby producing results visually accurate when compared to an FEM simulation. Second, EMU is efficient and scales much better than state‐of‐the‐art FEM with the number of elements in the mesh, and is more easily parallelizable. Third, EMU can handle heterogeneously stiff meshes with an arbitrary constitutive model, thus allowing it to simulate soft muscles, stiff tendons and even stiffer bones all within one unified system. These three key characteristics of EMU enable us to efficiently orchestrate muscle activated skeletal movements. We demonstrate the efficacy of our approach via a number of examples with tendons, muscles, bones and joints.
Structural Analogy from a Single Image Pair
(© 2021 Eurographics ‐ The European Association for Computer Graphics and John Wiley & Sons Ltd, 2021) Benaim, S.; Mokady, R.; Bermano, A.; Wolf, L.; Benes, Bedrich and Hauser, Helwig
The task of unsupervised image‐to‐image translation has seen substantial advancements in recent years through the use of deep neural networks. Typically, the proposed solutions learn the characterizing distribution of two large, unpaired collections of images, and are able to alter the appearance of a given image, while keeping its geometry intact. In this paper, we explore the capabilities of neural networks to understand image given only a single pair of images, and . We seek to generate images that are : that is, to generate an image that keeps the appearance and style of , but has a structural arrangement that corresponds to . The key idea is to map between image patches at different scales. This enables controlling the granularity at which analogies are produced, which determines the conceptual distinction between style and content. In addition to , our method can be used to generate high quality imagery in other conditional generation tasks utilizing images and only: guided image synthesis, style and texture transfer, text translation as well as video translation. Our code and additional results are available in
Time‐Warped Foveated Rendering for Virtual Reality Headsets
(© 2021 Eurographics ‐ The European Association for Computer Graphics and John Wiley & Sons Ltd, 2021) Franke, Linus; Fink, Laura; Martschinke, Jana; Selgrad, Kai; Stamminger, Marc; Benes, Bedrich and Hauser, Helwig
Rendering in real time for virtual reality headsets with high user immersion is challenging due to strict framerate constraints as well as due to a low tolerance for artefacts. Eye tracking‐based foveated rendering presents an opportunity to strongly increase performance without loss of perceived visual quality. To this end, we propose a novel foveated rendering method for virtual reality headsets with integrated eye tracking hardware. Our method comprises recycling pixels in the periphery by spatio‐temporally reprojecting them from previous frames. Artefacts and disocclusions caused by this reprojection are detected and re‐evaluated according to a confidence value that is determined by a newly introduced formalized perception‐based metric, referred to as confidence function. The foveal region, as well as areas with low confidence values, are redrawn efficiently, as the confidence value allows for the delicate regulation of hierarchical geometry and pixel culling. Hence, the average primitive processing and shading costs are lowered dramatically. Evaluated against regular rendering as well as established foveated rendering methods, our approach shows increased performance in both cases. Furthermore, our method is not restricted to static scenes and provides an acceleration structure for post‐processing passes.
ECHO: Extended Convolution Histogram of Orientations for Local Surface Description
(© 2021 Eurographics ‐ The European Association for Computer Graphics and John Wiley & Sons Ltd, 2021) Mitchel, Thomas W.; Rusinkiewicz, Szymon; Chirikjian, Gregory S.; Kazhdan, Michael; Benes, Bedrich and Hauser, Helwig
This paper presents a novel, highly distinctive and robust local surface feature descriptor. Our descriptor is predicated on a simple observation: instead of describing the points in the vicinity of a feature point relative to a reference frame at the feature point, all points in the region describe the feature point relative to their own frames. Isometry invariance is a byproduct of this construction. Our descriptor is derived relative to the extended convolution – a generalization of the standard convolution that allows the filter to adaptively transform as it passes over the domain. As such, we name our descriptor the Extended Convolution Histogram of Orientations (ECHO). It exhibits superior performance compared to popular surface descriptors in both feature matching and shape correspondence experiments. In particular, the ECHO descriptor is highly stable under near‐isometric deformations and remains distinctive under significant levels of noise, tessellation, complex deformations and the kinds of interference commonly found in real data.
Erratum: Adjustable Constrained Soft‐Tissue Dynamics
(© 2021 Eurographics ‐ The European Association for Computer Graphics and John Wiley & Sons Ltd, 2021) Benes, Bedrich and Hauser, Helwig
TopoAct: Visually Exploring the Shape of Activations in Deep Learning
(© 2021 Eurographics ‐ The European Association for Computer Graphics and John Wiley & Sons Ltd, 2021) Rathore, Archit; Chalapathi, Nithin; Palande, Sourabh; Wang, Bei; Benes, Bedrich and Hauser, Helwig
Deep neural networks such as GoogLeNet, ResNet, and BERT have achieved impressive performance in tasks such as image and text classification. To understand how such performance is achieved, we probe a trained deep neural network by studying neuron activations, i.e.combinations of neuron firings, at various layers of the network in response to a particular input. With a large number of inputs, we aim to obtain a global view of what neurons detect by studying their activations. In particular, we develop visualizations that show the shape of the activation space, the organizational principle behind neuron activations, and the relationships of these activations within a layer. Applying tools from topological data analysis, we present , a visual exploration system to study topological summaries of activation vectors. We present exploration scenarios using that provide valuable insights into learned representations of neural networks. We expect to give a topological perspective that enriches the current toolbox of neural network analysis, and to provide a basis for network architecture diagnosis and data anomaly detection.
A Modified Double Gyre with Ground Truth Hyperbolic Trajectories for Flow Visualization
(© 2021 Eurographics ‐ The European Association for Computer Graphics and John Wiley & Sons Ltd, 2021) Wolligandt, S.; Wilde, T.; Rössl, C.; Theisel, H.; Benes, Bedrich and Hauser, Helwig
The model of a flow by Shadden et al. is a standard benchmark data set for the computation of hyperbolic Lagrangian Coherent Structures (LCS) in flow data. While structurally extremely simple, it generates hyperbolic LCS of arbitrary complexity. Unfortunately, the does not come with a well‐defined ground truth: the location of hyperbolic LCS boundaries can only be approximated by numerical methods that usually involve the gradient of the flow map. We present a new benchmark data set that is a small but carefully designed modification of the , which comes with ground truth closed‐form hyperbolic trajectories. This allows for computing hyperbolic LCS boundaries by a simple particle integration without the consideration of the flow map gradient. We use these hyperbolic LCS as a ground truth solution for testing an existing numerical approach for extracting hyperbolic trajectories. In addition, we are able to construct hyperbolic LCS curves that are significantly longer than in existing numerical methods.
A Curvature and Density‐based Generative Representation of Shapes
(© 2021 Eurographics ‐ The European Association for Computer Graphics and John Wiley & Sons Ltd, 2021) Ye, Z.; Umetani, N.; Igarashi, T.; Hoffmann, T.; Benes, Bedrich and Hauser, Helwig
This paper introduces a generative model for 3D surfaces based on a representation of shapes with mean curvature and metric, which are invariant under rigid transformation. Hence, compared with existing 3D machine learning frameworks, our model substantially reduces the influence of translation and rotation. In addition, the local structure of shapes will be more precisely captured, since the curvature is explicitly encoded in our model. Specifically, every surface is first conformally mapped to a canonical domain, such as a unit disk or a unit sphere. Then, it is represented by two functions: the mean curvature half‐density and the vertex density, over this canonical domain. Assuming that input shapes follow a certain distribution in a latent space, we use the variational autoencoder to learn the latent space representation. After the learning, we can generate variations of shapes by randomly sampling the distribution in the latent space. Surfaces with triangular meshes can be reconstructed from the generated data by applying isotropic remeshing and spin transformation, which is given by Dirac equation. We demonstrate the effectiveness of our model on datasets of man‐made and biological shapes and compare the results with other methods.
Life cycle of SARS‐CoV‐2: from sketch to visualization in atomistic resolution
(© 2021 Eurographics ‐ The European Association for Computer Graphics and John Wiley & Sons Ltd, 2021) Benes, Bedrich and Hauser, Helwig
Functionality‐Driven Musculature Retargeting
(© 2021 Eurographics ‐ The European Association for Computer Graphics and John Wiley & Sons Ltd, 2021) Ryu, Hoseok; Kim, Minseok; Lee, Seungwhan; Park, Moon Seok; Lee, Kyoungmin; Lee, Jehee; Benes, Bedrich and Hauser, Helwig
We present a novel retargeting algorithm that transfers the musculature of a reference anatomical model to new bodies with different sizes, body proportions, muscle capability, and joint range of motion while preserving the functionality of the original musculature as closely as possible. The geometric configuration and physiological parameters of musculotendon units are estimated and optimized to adapt to new bodies. The range of motion around joints is estimated from a motion capture dataset and edited further for individual models. The retargeted model is simulation‐ready, so we can physically simulate muscle‐actuated motor skills with the model. Our system is capable of generating a wide variety of anatomical bodies that can be simulated to walk, run, jump and dance while maintaining balance under gravity. We will also demonstrate the construction of individualized musculoskeletal models from bi‐planar X‐ray images and medical examination.
Towards Light‐Weight Portrait Matting via Parameter Sharing
(© 2021 Eurographics ‐ The European Association for Computer Graphics and John Wiley & Sons Ltd, 2021) Dai, Yutong; Lu, Hao; Shen, Chunhua; Benes, Bedrich and Hauser, Helwig
Traditional portrait matting methods typically consist of a trimap estimation network and a matting network. Here, we propose a new light‐weight portrait matting approach, termed parameter‐sharing portrait matting (PSPM). Different from conventional portrait matting models where the encoder and decoder networks in two tasks are often separately designed, here a single encoder is employed for the two tasks in PSPM, while each task still has its task‐specific decoder. Thus, the role of the encoder is to extract semantic features and two decoders function as a bridge between low‐resolution feature maps generated by the encoder and high‐resolution feature maps for pixel‐wise classification/regression. In particular, three variants capable of implementing the parameter‐sharing portrait matting network are proposed and investigated, respectively. As demonstrated in our experiments, model capacity and computation costs can be reduced significantly, by up to and , respectively, with PSPM, whereas the matting accuracy only slightly deteriorates. In addition, qualitative and quantitative evaluations show that sharing the encoder is an effective way to achieve portrait matting with limited computational budgets, indicating a promising direction for applications of real‐time portrait matting on mobile devices.
Issue Information
(© 2021 Eurographics ‐ The European Association for Computer Graphics and John Wiley & Sons Ltd, 2021) Benes, Bedrich and Hauser, Helwig

Browse

Browsing 40-Issue 1 by Issue Date

Results Per Page

Sort Options