37-Issue 7

Permanent URI for this collection

https://diglib.eg.org/handle/10.2312/2632664

Browse

Now showing 1 - 20 of 43

Modeling Fonts in Context: Font Prediction on Web Designs
(The Eurographics Association and John Wiley & Sons Ltd., 2018) Zhao, Nanxuan; Cao, Ying; Lau, Rynson W. H.; Fu, Hongbo and Ghosh, Abhijeet and Kopf, Johannes
Web designers often carefully select fonts to fit the context of a web design to make the design look aesthetically pleasing and effective in communication. However, selecting proper fonts for a web design is a tedious and time-consuming task, as each font has many properties, such as font face, color, and size, resulting in a very large search space. In this paper, we aim to model fonts in context, by studying a novel and challenging problem of predicting fonts that match a given web design. To this end, we propose a novel, multi-task deep neural network to jointly predict font face, color and size for each text element on a web design, by considering multi-scale visual features and semantic tags of the web design. To train our model, we have collected a CTXFont dataset, which consists of 1k professional web designs, with labeled font properties. Experiments show that our model outperforms the baseline methods, achieving promising qualitative and quantitative results on the font selection task. We also demonstrate the usefulness of our method in a font selection task via a user study.
Feature Generation for Adaptive Gradient-Domain Path Tracing
(The Eurographics Association and John Wiley & Sons Ltd., 2018) Back, Jonghee; Yoon, Sung-Eui; Moon, Bochang; Fu, Hongbo and Ghosh, Abhijeet and Kopf, Johannes
In this paper, we propose a new technique to incorporate recent adaptive rendering approaches built upon local regression theory into a gradient-domain path tracing framework, in order to achieve high-quality rendering results. Our method aims to reduce random artifacts introduced by random sampling on image colors and gradients. Our high-level approach is to identify a feature image from noisy gradients, and pass the image to an existing local regression based adaptive method so that adaptive sampling and reconstruction using our feature can boost the performance of gradient-domain rendering. To fulfill our idea, we derive an ideal feature in the form of image gradients and propose an estimation process for the ideal feature in the presence of noise in image gradients. We demonstrate that our integrated adaptive solution leads to performance improvement for a gradient-domain path tracer, by seamlessly incorporating recent adaptive sampling and reconstruction strategies through our estimated feature.
GPU-based Polynomial Finite Element Matrix Assembly for Simplex Meshes
(The Eurographics Association and John Wiley & Sons Ltd., 2018) Mueller-Roemer, Johannes Sebastian; Stork, André; Fu, Hongbo and Ghosh, Abhijeet and Kopf, Johannes
In this paper, we present a matrix assembly technique for arbitrary polynomial order finite element simulations on simplex meshes for graphics processing units (GPU). Compared to the current state of the art in GPU-based matrix assembly, we avoid the need for an intermediate sparse matrix and perform assembly directly into the final, GPU-optimized data structure. Thereby, we avoid the resulting 180% to 600% memory overhead, depending on polynomial order, and associated allocation time, while simplifying the assembly code and using a more compact mesh representation. We compare our method with existing algorithms and demonstrate significant speedups.
Local and Hierarchical Refinement for Subdivision Gradient Meshes
(The Eurographics Association and John Wiley & Sons Ltd., 2018) Verstraaten, Teun W.; Kosinka, Jiri; Fu, Hongbo and Ghosh, Abhijeet and Kopf, Johannes
Gradient mesh design tools allow users to create detailed scalable images, traditionally through the creation and manipulation of a (dense) mesh with regular rectangular topology. Through recent advances it is now possible to allow gradient meshes to have arbitrary manifold topology, using a modified Catmull-Clark subdivision scheme to define the resultant geometry and colour [LKSD17]. We present two novel methods to allow local and hierarchical refinement of both colour and geometry for such subdivision gradient meshes. Our methods leverage the mesh properties that the particular subdivision scheme ensures. In both methods, the artists enjoy all the standard capabilities of manipulating the mesh and the associated colour gradients at the coarsest level as well as locally at refined levels. Further novel features include interpolation of both position and colour of the vertices of the input meshes, local detail follows coarser-level edits, and support for sharp colour transitions, all at any level in the hierarchy offered by subdivision.
Uncut Aerial Video via a Single Sketch
(The Eurographics Association and John Wiley & Sons Ltd., 2018) Yang, Hao; Xie, Ke; Huang, Shengqiu; Huang, Hui; Fu, Hongbo and Ghosh, Abhijeet and Kopf, Johannes
Nowadays UAV filming is getting popular, more and more stunning aerial videos appearing online. Nonetheless, making a good uncut aerial video with only one-long-shot for the large-scale outdoor scenes is still quite challenging, no many eye-catching pieces available yet. It requires users to have both consummate drone controlling skill and good perception of filming aesthetics. If totally manual, the user has to simultaneously adjust the drone position and the mounted camera orientation during the whole flyby while trying to keep all operation changes executed smoothly. Recent research has proposed a number of planning tools for automatic or semi-automatic aerial videography, however, most requires rather complex user inputs and heavy computations. In this paper, we propose a user-friendly system designed to simplify the input and automatically generate continuous camera moves to capture compelling aerial videos that users prefer to see without any post cutting or editing. Assume a rough 2.5D scene model that includes all the regions of interest are available, users are only required to casually draw a single sketch on the 2D map. Our system will analyze this rough sketch input, compute the corresponding quality views in 3D safe flying zone, and then create a globally optimal camera trajectory passing through regions of user interest via solving a combinatorial problem. At end, we optimize the drone flying speed locally to make the resulting aerial videos more visually pleasing.
Mumford-Shah Mesh Processing using the Ambrosio-Tortorelli Functional
(The Eurographics Association and John Wiley & Sons Ltd., 2018) Bonneel, Nicolas; Coeurjolly, David; Gueth, Pierre; Lachaud, Jacques-Olivier; Fu, Hongbo and Ghosh, Abhijeet and Kopf, Johannes
The Mumford-Shah functional approximates a function by a piecewise smooth function. Its versatility makes it ideal for tasks such as image segmentation or restoration, and it is now a widespread tool of image processing. Recent work has started to investigate its use for mesh segmentation and feature lines detection, but we take the stance that the power of this functional could reach far beyond these tasks and integrate the everyday mesh processing toolbox. In this paper, we discretize an Ambrosio-Tortorelli approximation via a Discrete Exterior Calculus formulation. We show that, combined with a new shape optimization routine, several mesh processing problems can be readily tackled within the same framework. In particular, we illustrate applications in mesh denoising, normal map embossing, mesh inpainting and mesh segmentation.
Directing the Photography: Combining Cinematic Rules, Indirect Light Controls and Lighting-by-Example
(The Eurographics Association and John Wiley & Sons Ltd., 2018) Galvane, Quentin; Lino, Christophe; Christie, Marc; Cozot, Rémi; Fu, Hongbo and Ghosh, Abhijeet and Kopf, Johannes
The placement of lights in a 3D scene is a technical and artistic task that requires time and trained skills. Most 3D modelling tools only provide a direct control of light sources, through the manipulation of parameters such as size, location, flux (the perceived power of light) or opening angle (the light frustum). Approaches have been relying on automated or semi-automated techniques to relieve users from such low-level manipulations at the expense of an important computational cost. In this paper, guided by discussions with experts in scene and object lighting, we propose an indirect control of area light sources. We first formalize the classical 3-point lighting design principle (key-light, fill-lights and back/rim-lights) in a parametric model. Given a key-light placed in the scene, we then provide a computational approach to (i) automatically compute the position and size of fill-lights and back/rim-lights by analyzing the geometry of 3D character, and (ii) automatically compute the flux and size of key, fill and back/rim lights, given a sample reference image in a computationally efficient way. Results demonstrate the benefits of the approach on the quick lighting of 3D characters, and further demonstrate the feasibility of interactive control of multiple lights through image features.
Unsupervised Deep Single-Image Intrinsic Decomposition using Illumination-Varying Image Sequences
(The Eurographics Association and John Wiley & Sons Ltd., 2018) Lettry, Louis; Vanhoey, Kenneth; Van Gool, Luc; Fu, Hongbo and Ghosh, Abhijeet and Kopf, Johannes
Machine learning based Single Image Intrinsic Decomposition (SIID) methods decompose a captured scene into its albedo and shading images by using the knowledge of a large set of known and realistic ground truth decompositions. Collecting and annotating such a dataset is an approach that cannot scale to sufficient variety and realism. We free ourselves from this limitation by training on unannotated images. Our method leverages the observation that two images of the same scene but with different lighting provide useful information on their intrinsic properties: by definition, albedo is invariant to lighting conditions, and cross-combining the estimated albedo of a first image with the estimated shading of a second one should lead back to the second one's input image. We transcribe this relationship into a siamese training scheme for a deep convolutional neural network that decomposes a single image into albedo and shading. The siamese setting allows us to introduce a new loss function including such cross-combinations, and to train solely on (time-lapse) images, discarding the need for any ground truth annotations. As a result, our method has the good properties of i) taking advantage of the time-varying information of image sequences in the (pre-computed) training step, ii) not requiring ground truth data to train on, and iii) being able to decompose single images of unseen scenes at runtime. To demonstrate and evaluate our work, we additionally propose a new rendered dataset containing illumination-varying scenes and a set of quantitative metrics to evaluate SIID algorithms. Despite its unsupervised nature, our results compete with state of the art methods, including supervised and non data-driven methods.
A New Uniform Format for 360 VR Videos
(The Eurographics Association and John Wiley & Sons Ltd., 2018) Guo, Juan; Pei, Qikai K.; Ma, Guilong L.; Liu, Li; Zhang, Xinyu Y.; Fu, Hongbo and Ghosh, Abhijeet and Kopf, Johannes
Recent breakthroughs in VR technologies, especially in economic VR headsets and massive smartphones are creating a fastgrowing demand for 3D immersive VR content. 360 VR videos record a surrounding environment in every direction and give users a fully immersive experience. Thanks to a ton of 360 cameras that launched in the past years, 360 video content creation is exploding and 360 VR videos are becoming a new video standard in the digital industry. When ERP and CMP are perhaps the most prevalent projection and packing layout for storing 360 VR videos, they have severe projection distortion, internal discontinuous seams or disadvantages in aspect ratio. We introduce a new format for packing and storing 360 VR videos using two stage mappings. Hemispheres are seamlessly and uniformly mapped onto squares. Two respective squares are stitched to form a rectangle with the aspect ratio 2 : 1. Our approach is able to avoid internal discontinuity and generate uniform pixel distribution, while keeping the aspect ratio close to the majority standard aspect ratio of 16 : 9.
Shape and Pose Estimation for Closely Interacting Persons Using Multi-view Images
(The Eurographics Association and John Wiley & Sons Ltd., 2018) Li, Kun; Jiao, Nianhong; Liu, Yebin; Wang, Yangang; Yang, Jingyu; Fu, Hongbo and Ghosh, Abhijeet and Kopf, Johannes
Multi-person pose and shape estimation is very challenging, especially when the persons have close interactions. Existing methods only work well when people are well spaced out in the captured images. However, close interaction among people is very common in real life, which is more challenge due to complex articulation, frequent occlusion and inherent ambiguities. We present a fully-automatic markerless motion capture method to simultaneously estimate 3D poses and shapes of closely interacting people from multi-view sequences. We first predict the 2D joints for each person in an image, and then design a spatio-temporal tracker for multi-person pose tracking based on multi-view videos. Finally, we estimate 3D poses and shapes of all the persons with multi-view constraints using a skinned multi-person linear model (SMPL). Experimental results demonstrate that our method achieves fast but accurate pose and shape estimation results for multi-person close interaction cases. Compared with existing methods, our method does not need pre-segmentation for each person and manual intervention, which greatly reduces the complexity of the system including time complexity and system processing complexity.
Reformulating Hyperelastic Materials with Peridynamic Modeling
(The Eurographics Association and John Wiley & Sons Ltd., 2018) Xu, Liyou; He, Xiaowei; Chen, Wei; Li, Sheng; Wang, Guoping; Fu, Hongbo and Ghosh, Abhijeet and Kopf, Johannes
Peridynamics is a formulation of the classical elastic theory that is targeted at simulating deformable objects with discontinuities, especially fractures. Till now, there are few studies that have been focused on how to model general hyperelastic materials with peridynamics. In this paper, we target at proposing a general strain energy function of hyperelastic materials for peridynamics. To get an intuitive model that can be easily controlled, we formulate the strain energy density function as a function parameterized by the dilatation and bond stretches, which can be decomposed into multiple one-dimensional functions independently. To account for nonlinear material behaviors, we also propose a set of nonlinear basis functions to help design a nonlinear strain energy function more easily. For an anisotropic material, we additionally introduce an anisotropic kernel to control the elastic behavior for each bond independently. Experiments show that our model is flexible enough to approximately regenerate various hyperelastic materials in classical elastic theory, including St.Venant-Kirchhoff and Neo-Hookean materials.
Subdivision Schemes With Optimal Bounded Curvature Near Extraordinary Vertices
(The Eurographics Association and John Wiley & Sons Ltd., 2018) Ma, Yue; Ma, Weiyin; Fu, Hongbo and Ghosh, Abhijeet and Kopf, Johannes
We present a novel method to construct subdivision stencils near extraordinary vertices with limit surfaces having optimal bounded curvature at extraordinary positions. With the proposed method, subdivision stencils for newly inserted and updated vertices near extraordinary vertices are first constructed to ensure subdivision with G1 continuity and bounded curvature at extraordinary positions. The remaining degrees of freedom of the constructed subdivision stencils are further used to optimize the eigenbasis functions corresponding to the subsubdominant eigenvalues of the subdivision with respect to G2 continuity constraints. We demonstrate the method by replacing subdivision stencils near extraordinary vertices for Catmull-Clark subdivision and compare the results with the original Catmull-Clark subdivision and previous tuning schemes known with small curvature variation near extraordinary positions. The results show that the proposed method produces subdivision schemes with better or comparable curvature behavior around extraordinary vertices with comparatively simple subdivision stencils.
Generative Adversarial Image Super-Resolution Through Deep Dense Skip Connections
(The Eurographics Association and John Wiley & Sons Ltd., 2018) Zhu, Xiaobin; Li, Zhuangzi; Zhang, Xiaoyu; Li, Haisheng; Xue, Ziyu; Wang, Lei; Fu, Hongbo and Ghosh, Abhijeet and Kopf, Johannes
Recently, image super-resolution works based on Convolutional Neural Networks (CNNs) and Generative Adversarial Nets (GANs) have shown promising performance. However, these methods tend to generate blurry and over-smoothed super-resolved (SR) images, due to the incomplete loss function and powerless architectures of networks. In this paper, a novel generative adversarial image super-resolution through deep dense skip connections (GSR-DDNet), is proposed to solve the above-mentioned problems. It aims to take advantage of GAN's ability of modeling data distributions, so that GSR-DDNet can select informative feature representation and model the mapping across the low-quality and high-quality images in an adversarial way. The pipeline of the proposed method consists of three main components: 1) The generator of a novel dense skip connection network with the deep structure for learning robust mapping function is proposed to generate SR images from low-resolution images; 2) The feature extraction network based on VGG-19 is adopted to capture high frequency feature maps for content loss; and 3) The discriminator with Wasserstein distance is adopted to identify the overall style of SR and ground-truth images. Experiments conducted on four publicly available datasets demonstrate the superiority against the state-of-the-art methods.
Decomposing Images into Layers with Advanced Color Blending
(The Eurographics Association and John Wiley & Sons Ltd., 2018) Koyama, Yuki; Goto, Masataka; Fu, Hongbo and Ghosh, Abhijeet and Kopf, Johannes
Digital paintings are often created by compositing semi-transparent layers using various advanced color-blend modes, such as ''color-burn,'' ''multiply,'' and ''screen,'' which can produce interesting non-linear color effects. We propose a method of decomposing an input image into layers with such advanced color blending. Unlike previous layer-decomposition methods, which typically support only linear color-blend modes, ours can handle any user-specified color-blend modes. To enable this, we generalize a previous color-unblending formulation, in which only a specific layering model was considered. We also introduce several techniques for adapting our generalized formulation to practical use, such as the post-processing for refining smoothness. Our method lets users explore possible decompositions to find the one that matches for their purposes by manipulating the target color-blend mode and desired color distribution for each layer, as well as the number of layers. Thus, the output of our method is a layered, easily editable image composition organized in a way that digital artists are familiar with. Our method is useful for remixing existing illustrations, flexibly editing single-layer paintings, and bringing physically painted media (e.g., oil paintings) into a digital workflow.
Sit & Relax: Interactive Design of Body-Supporting Surfaces
(The Eurographics Association and John Wiley & Sons Ltd., 2018) Leimer, Kurt; Birsak, Michael; Rist, Florian; Musialski, Przemyslaw; Fu, Hongbo and Ghosh, Abhijeet and Kopf, Johannes
We propose a novel method for interactive design of well-fitting body-supporting surfaces that is driven by the pressure distribution on the body's surface. Our main contribution is an interactive modeling system that utilizes captured body poses and computes an importance field that is proportional to the pressure distribution on the body for a given pose. This distribution indicates where the body should be supported in order to easily hold a particular pose, which is one of the measures of comfortable sitting. Using our approximation, we propose the entire workflow for interactive design of C2 smooth surfaces which serve as seats, or generally, as body supporting furniture for comfortable sitting. Finally, we also provide a design tool for RHINOCEROS/GRASSHOPPER that allows for interactive creation of single designs or entire multi-person sitting scenarios. We also test the tool with design students and present several results. Our method aims at interactive design in order to help designers to create appropriate surfaces digitally without additional empirical design passes.
Improved Use of LOP for Curve Skeleton Extraction
(The Eurographics Association and John Wiley & Sons Ltd., 2018) Li, Lei; Wang, Wencheng; Fu, Hongbo and Ghosh, Abhijeet and Kopf, Johannes
It remains a challenge to robustly and rapidly extract high quality curve skeletons from 3D models of closed surfaces, especially when there are nearby surface sheets. In this paper, we address this challenge by improving the use of LOP (Locally Optimal Projection) to adaptively contract medial surfaces of 3D models. LOP was originally designed to optimize a raw scanned point cloud to its corresponding geometry surface. It has the effect of contraction, and the contraction amplitude is controlled by a support radius. Our improvements are twofold. First, we constrain the LOP operator applied in the 2D medial surface instead of in the 3D space and take a local region growing strategy to find neighborhoods for implementing LOP. Thus, we avoid interference between disconnected surface parts and accelerate the process due to the reduced search space. Second, we adaptively adjust the support radii to have different parts of the medial surface contracted adaptively and synchronously for generating connected skeletal curves. In this paper, we demonstrate that our method allows for each part of the medial surface to be contracted symmetrically to its center line and is insensitive to surface noises. Thus, with our method, centered and connected high quality curve skeletons can be extracted robustly and rapidly, even for models with nearby surface sheets. Experimental results highlight the effectiveness and high efficiency of the method, even for noisy and topologically complex models, making it superior to other state-of-the-art methods.
FashionGAN: Display your fashion design using Conditional Generative Adversarial Nets
(The Eurographics Association and John Wiley & Sons Ltd., 2018) Cui, Yi Rui; Liu, Qi; Gao, Cheng Ying; Su, Zhuo; Fu, Hongbo and Ghosh, Abhijeet and Kopf, Johannes
Virtual garment display plays an important role in fashion design for it can directly show the design effect of the garment without having to make a sample garment like traditional clothing industry. In this paper, we propose an end-to-end virtual garment display method based on Conditional Generative Adversarial Networks. Different from existing 3D virtual garment methods which need complex interactions and domain-specific user knowledge, our method only need users to input a desired fashion sketch and a specified fabric image then the image of the virtual garment whose shape and texture are consistent with the input fashion sketch and fabric image can be shown out quickly and automatically. Moreover, it can also be extended to contour images and garment images, which further improves the reuse rate of fashion design. Compared with the existing image-to-image methods, the quality of images generated by our method is better in terms of color and shape.
Deep Video Stabilization Using Adversarial Networks
(The Eurographics Association and John Wiley & Sons Ltd., 2018) Xu, Sen-Zhe; Hu, Jun; Wang, Miao; Mu, Tai-Jiang; Hu, Shi-Min; Fu, Hongbo and Ghosh, Abhijeet and Kopf, Johannes
Video stabilization is necessary for many hand-held shot videos. In the past decades, although various video stabilization methods were proposed based on the smoothing of 2D, 2.5D or 3D camera paths, hardly have there been any deep learning methods to solve this problem. Instead of explicitly estimating and smoothing the camera path, we present a novel online deep learning framework to learn the stabilization transformation for each unsteady frame, given historical steady frames. Our network is composed of a generative network with spatial transformer networks embedded in different layers, and generates a stable frame for the incoming unstable frame by computing an appropriate affine transformation. We also introduce an adversarial network to determine the stability of a piece of video. The network is trained directly using the pair of steady and unsteady videos. Experiments show that our method can produce similar results as traditional methods, moreover, it is capable of handling challenging unsteady video of low quality, where traditional methods fail, such as video with heavy noise or multiple exposures. Our method runs in real time, which is much faster than traditional methods.
Curvature Continuity Conditions Between Adjacent Toric Surface Patches
(The Eurographics Association and John Wiley & Sons Ltd., 2018) Sun, Lanyin; Zhu, Chungang; Fu, Hongbo and Ghosh, Abhijeet and Kopf, Johannes
Toric surface patch is the multi-sided generalization of classical Bézier surface patch. Geometric continuity of the parametric surface patches plays a crucial role in geometric modeling. In this paper, the necessary and sufficient conditions of curvature continuity between toric surface patches are illustrated with the theory of toric degeneration. Furthermore, some practical sufficient conditions of curvature continuity of toric surface patches are also developed.
Non-Local Low-Rank Normal Filtering for Mesh Denoising
(The Eurographics Association and John Wiley & Sons Ltd., 2018) Li, Xianzhi; Zhu, Lei; Fu, Chi-Wing; Heng, Pheng-Ann; Fu, Hongbo and Ghosh, Abhijeet and Kopf, Johannes
This paper presents a non-local low-rank normal filtering method for mesh denoising. By exploring the geometric similarity between local surface patches on 3D meshes in the form of normal fields, we devise a low-rank recovery model that filters normal vectors by means of patch groups. In summary, our method has the following key contributions. First, we present the guided normal patch covariance descriptor to analyze the similarity between patches. Second, we pack normal vectors on similar patches into the normal-field patch-group (NPG) matrix for rank analysis. Third, we formulate mesh denoising as a low-rank matrix recovery problem based on the prior that the rank of the NPG matrix is high for raw meshes with noise, but can be significantly reduced for denoised meshes, whose normal vectors across similar patches should be more strongly correlated. Furthermore, we devise an objective function based on an improved truncated 'gamma' norm, and derive an optimization procedure using the alternative direction method of multipliers and iteratively re-weighted least squares techniques.We conducted several experiments to evaluate our method using various 3D models, and compared our results against several state-of-the-art methods. Experimental results show that our method consistently outperforms other methods and better preserves the fine details.

Browse

Browsing 37-Issue 7 by Issue Date

Results Per Page

Sort Options