43-Issue 7

Permanent URI for this collection

https://diglib.eg.org/handle/10.2312/3607047

Browse

Now showing 1 - 20 of 57

MISNeR: Medical Implicit Shape Neural Representation for Image Volume Visualisation
(The Eurographics Association and John Wiley & Sons Ltd., 2024) Jin, Ge; Jung, Younhyun; Bi, Lei; Kim, Jinman; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
Three-dimensional visualisation of mesh reconstruction of the medical images is commonly used for various clinical applications including pre / post-surgical planning. Such meshes are conventionally generated by extracting the surface from volumetric segmentation masks. Therefore, they have inherent limitations of staircase artefacts due to their anisotropic voxel dimensions. The time-consuming process for manual refinement to remove artefacts and/or the isolated regions further adds to these limitations. Methods for directly generating meshes from volumetric data by template deformation are often limited to simple topological structures, and methods that use implicit functions for continuous surfaces, do not achieve the level of mesh reconstruction accuracy when compared to segmentation-based methods. In this study, we address these limitations by combining the implicit function representation with a multi-level deep learning architecture. We introduce a novel multi-level local feature sampling component which leverages the spatial features for the implicit function regression to enhance the segmentation result. We further introduce a shape boundary estimator that accelerates the explicit mesh reconstruction by minimising the number of the signed distance queries during model inference. The result is a multi-level deep learning network that directly regresses the implicit function from medical image volumes to a continuous surface model, which can be used for mesh reconstruction from arbitrary high volume resolution to minimise staircase artefacts. We evaluated our method using pelvic computed tomography (CT) dataset from two public sources with varying z-axis resolutions. We show that our method minimised the staircase artefacts while achieving comparable results in surface accuracy when compared to the state-of-the-art segmentation algorithms. Furthermore, our method was 9 times faster in volume reconstruction than comparable implicit shape representation networks.
SOD-diffusion: Salient Object Detection via Diffusion-Based Image Generators
(The Eurographics Association and John Wiley & Sons Ltd., 2024) Zhang, Shuo; Huang, Jiaming; Chen, Shizhe; Wu, Yan; Hu, Tao; Liu, Jing; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
Salient Object Detection (SOD) is a challenging task that aims to precisely identify and segment the salient objects. However, existing SOD methods still face challenges in making explicit predictions near the edges and often lack end-to-end training capabilities. To alleviate these problems, we propose SOD-diffusion, a novel framework that formulates salient object detection as a denoising diffusion process from noisy masks to object masks. Specifically, object masks diffuse from ground-truth masks to random distribution in latent space, and the model learns to reverse this noising process to reconstruct object masks. To enhance the denoising learning process, we design an attention feature interaction module (AFIM) and a specific fine-tuning protocol to integrate conditional semantic features from the input image with diffusion noise embedding. Extensive experiments on five widely used SOD benchmark datasets demonstrate that our proposed SOD-diffusion achieves favorable performance compared to previous well-established methods. Furthermore, leveraging the outstanding generalization capability of SOD-diffusion, we applied it to publicly available images, generating high-quality masks that serve as an additional SOD benchmark testset.
A Surface-based Appearance Model for Pennaceous Feathers
(The Eurographics Association and John Wiley & Sons Ltd., 2024) Padrón-Griffe, Juan Raúl; Lanza, Dario; Jarabo, Adrian; Muñoz, Adolfo; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
The appearance of a real-world feather results from the complex interaction of light with its multi-scale biological structure, including the central shaft, branching barbs, and interlocking barbules on those barbs. In this work, we propose a practical surface-based appearance model for feathers. We represent the far-field appearance of feathers using a BSDF that implicitly represents the light scattering from the main biological structures of a feather, such as the shaft, barb and barbules. Our model accounts for the particular characteristics of feather barbs such as the non-cylindrical cross-sections and the scattering media via a numerically-based BCSDF. To model the relative visibility between barbs and barbules, we derive a masking term for the differential projected areas of the different components of the feather's microgeometry, which allows us to analytically compute the masking between barbs and barbules. As opposed to previous works, our model uses a lightweight representation of the geometry based on a 2D texture, and does not require explicitly representing the barbs as curves. We show the flexibility and potential of our appearance model approach to represent the most important visual features of several pennaceous feathers.
DSGI-Net: Density-based Selective Grouping Point Cloud Learning Network for Indoor Scene
(The Eurographics Association and John Wiley & Sons Ltd., 2024) Wen, Xin; Duan, Yao; Xu, Kai; Zhu, Chenyang; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
Indoor scene point clouds exhibit diverse distributions and varying levels of sparsity, characterized by more intricate geometry and occlusion compared to outdoor scenes or individual objects. Despite recent advancements in 3D point cloud analysis introducing various network architectures, there remains a lack of frameworks tailored to the unique attributes of indoor scenarios. To address this, we propose DSGI-Net, a novel indoor scene point cloud learning network that can be integrated into existing models. The key innovation of this work is selectively grouping more informative neighbor points in sparse regions and promoting semantic consistency of the local area where different instances are in proximity but belong to distinct categories. Furthermore, our method encodes both semantic and spatial relationships between points in local regions to reduce the loss of local geometric details. Extensive experiments on the ScanNetv2, SUN RGB-D, and S3DIS indoor scene benchmarks demonstrate that our method is straightforward yet effective.
TempDiff: Enhancing Temporal-awareness in Latent Diffusion for Real-World Video Super-Resolution
(The Eurographics Association and John Wiley & Sons Ltd., 2024) Jiang, Qin; Wang, Qing Lin; Chi, Li Hua; Chen, Xin Hai; Zhang, Qing Yang; Zhou, Richard; Deng, Zheng Qiu; Deng, Jin Sheng; Tang, Bin Bing; Lv, Shao He; Liu, Jie; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
Latent diffusion models (LDMs) have demonstrated remarkable success in generative modeling. It is promising to leverage the potential of diffusion priors to enhance performance in image and video tasks. However, applying LDMs to video superresolution (VSR) presents significant challenges due to the high demands for realistic details and temporal consistency in generated videos, exacerbated by the inherent stochasticity in the diffusion process. In this work, we propose a novel diffusionbased framework, Temporal-awareness Latent Diffusion Model (TempDiff), specifically designed for real-world video superresolution, where degradations are diverse and complex. TempDiff harnesses the powerful generative prior of a pre-trained diffusion model and enhances temporal awareness through the following mechanisms: 1) Incorporating temporal layers into the denoising U-Net and VAE-Decoder, and fine-tuning these added modules to maintain temporal coherency; 2) Estimating optical flow guidance using a pre-trained flow net for latent optimization and propagation across video sequences, ensuring overall stability in the generated high-quality video. Extensive experiments demonstrate that TempDiff achieves compelling results, outperforming state-of-the-art methods on both synthetic and real-world VSR benchmark datasets. Code will be available at https://github.com/jiangqin567/TempDiff
Cinematic Gaussians: Real-Time HDR Radiance Fields with Depth of Field
(The Eurographics Association and John Wiley & Sons Ltd., 2024) Wang, Chao; Wolski, Krzysztof; Kerbl, Bernhard; Serrano, Ana; Bemama, Mojtaba; Seidel, Hans-Peter; Myszkowski, Karol; Leimkühler, Thomas; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
Radiance field methods represent the state of the art in reconstructing complex scenes from multi-view photos. However, these reconstructions often suffer from one or both of the following limitations: First, they typically represent scenes in low dynamic range (LDR), which restricts their use to evenly lit environments and hinders immersive viewing experiences. Secondly, their reliance on a pinhole camera model, assuming all scene elements are in focus in the input images, presents practical challenges and complicates refocusing during novel-view synthesis. Addressing these limitations, we present a lightweight method based on 3D Gaussian Splatting that utilizes multi-view LDR images of a scene with varying exposure times, apertures, and focus distances as input to reconstruct a high-dynamic-range (HDR) radiance field. By incorporating analytical convolutions of Gaussians based on a thin-lens camera model as well as a tonemapping module, our reconstructions enable the rendering of HDR content with flexible refocusing capabilities. We demonstrate that our combined treatment of HDR and depth of field facilitates real-time cinematic rendering, outperforming the state of the art.
Color-Accurate Camera Capture with Multispectral Illumination and Multiple Exposures
(The Eurographics Association and John Wiley & Sons Ltd., 2024) Gao, Hongyun; Mantiuk, Rafal K.; Finlayson, Graham D.; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
Cameras cannot capture the same colors as those seen by the human eye because the eye and the cameras' sensors differ in their spectral sensitivity. To obtain a plausible approximation of perceived colors, the camera's Image Signal Processor (ISP) employs a color correction step. However, even advanced color correction methods cannot solve this underdetermined problem, and visible color inaccuracies are always present. Here, we explore an approach in which we can capture accurate colors with a regular camera by optimizing the spectral composition of the illuminant and capturing one or more exposures. We jointly optimize for the signal-to-noise ratio and for the color accuracy irrespective of the spectral composition of the scene. One or more images captured under controlled multispectral illuminants are then converted into a color-accurate image as seen under the standard illuminant of D65. Our optimization allows us to reduce the color error by 20-60% (in terms of CIEDE 2000), depending on the number of exposures and camera type. The method can be used in applications in which illumination can be controlled, and high colour accuracy is required, such as product photography or with a multispectral camera flash. The code is available at https://github.com/gfxdisp/multispectral_color_correction.
Disk B-spline on S2: A Skeleton-based Region Representation on 2-Sphere
(The Eurographics Association and John Wiley & Sons Ltd., 2024) Zheng, Chunhao; Zhao, Yuming; Wu, Zhongke; Wang, Xingce; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
Due to the widespread applications of 2-dimensional spherical designs, there has been an increasing requirement of modeling on the S2 manifold in recent years. Due to the non-Euclidean nature of the sphere, it has some challenges to find a method to represent 2D regions on S2 manifold. In this paper, a skeleton-based representation method of regions on S2, disk B-spline(DBSC) on S2 is proposed. Firstly, we give the definition and basic algorithms of DBSC on S2. Then we provide the calculation method of DBSC on S2, which includes calculating the boundary points, internal points and their corresponding derivatives. Based on that, we give some modeling methods of DBSC on S2, including approximation, deformation. In the end, some stunning application examples of DBSC on S2 are shown. This work lays a theoretical foundation for further applications of DBSC on S2.
Strictly Conservative Neural Implicits
(The Eurographics Association and John Wiley & Sons Ltd., 2024) Ludwig, Ingmar; Campen, Marcel; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
We describe a method to convert 3D shapes into neural implicit form such that the shape is approximated in a guaranteed conservative manner. This means the input shape is strictly contained inside the neural implicit or, alternatively, vice versa. Such conservative approximations are of interest in a variety of applications, including collision detection, occlusion culling, or intersection testing. Our approach is the first to guarantee conservativeness in this context of neural implicits. We support input given as mesh, voxel set, or implicit function. Adaptive affine arithmetic is employed in the neural network fitting process, enabling the reasoning over infinite sets of points despite using a finite set of training data. Combined with an interior point style optimization approach this yields the desired guarantee.
Faster Ray Tracing through Hierarchy Cut Code
(The Eurographics Association and John Wiley & Sons Ltd., 2024) Xiang, WeiLai; Liu, FengQi; Tan, Zaonan; Li, Dan; Xu, PengZhan; Liu, MeiZhi; Kou, QiLong; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
We propose a novel ray reordering technique designed to accelerate the ray tracing process by encoding and sorting rays prior to traversal. Our method, called ''hierarchy cut code'', involves encoding rays based on the cuts of the hierarchical acceleration structure, rather than relying solely on spatial coordinates. This approach allows for a more effective adaptation to the acceleration structure, resulting in a more reliable and efficient encoding outcome. Furthermore, our research identifies ''bounding drift'' as a major obstacle in achieving better acceleration effects using longer sorting keys in existing reordering methods. Fortunately, our hierarchy cut code successfully overcomes this issue, providing improved performance in ray tracing. Experimental results demonstrate the effectiveness of our approach, showing up to a 1.81 times faster secondary ray tracing compared to existing methods. These promising results highlight the potential for further enhancement in the acceleration effect of reordering techniques, warranting further exploration and research in this exciting field.
CustomSketching: Sketch Concept Extraction for Sketch-based Image Synthesis and Editing
(The Eurographics Association and John Wiley & Sons Ltd., 2024) Xiao, Chufeng; Fu, Hongbo; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
Personalization techniques for large text-to-image (T2I) models allow users to incorporate new concepts from reference images. However, existing methods primarily rely on textual descriptions, leading to limited control over customized images and failing to support fine-grained and local editing (e.g., shape, pose, and details). In this paper, we identify sketches as an intuitive and versatile representation that can facilitate such control, e.g., contour lines capturing shape information and flow lines representing texture. This motivates us to explore a novel task of sketch concept extraction: given one or more sketch-image pairs, we aim to extract a special sketch concept that bridges the correspondence between the images and sketches, thus enabling sketch-based image synthesis and editing at a fine-grained level. To accomplish this, we introduce CustomSketching, a two-stage framework for extracting novel sketch concepts via few-shot learning. Considering that an object can often be depicted by a contour for general shapes and additional strokes for internal details, we introduce a dual-sketch representation to reduce the inherent ambiguity in sketch depiction. We employ a shape loss and a regularization loss to balance fidelity and editability during optimization. Through extensive experiments, a user study, and several applications, we show our method is effective and superior to the adapted baselines.
CrystalNet: Texture-Aware Neural Refraction Baking for Global Illumination
(The Eurographics Association and John Wiley & Sons Ltd., 2024) Zhang, Ziyang; Simo-Serra, Edgar; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
Neural rendering bakes global illumination and other computationally costly effects into the weights of a neural network, allowing to efficiently synthesize photorealistic images without relying on path tracing. In neural rendering approaches, G-buffers obtained from rasterization through direct rendering provide information regarding the scene such as position, normal, and textures to the neural network, achieving accurate and stable rendering quality in real-time. However, due to the use of G-buffers, existing methods struggle to accurately render transparency and refraction effects, as G-buffers do not capture any ray information from multiple light ray bounces. This limitation results in blurriness, distortions, and loss of detail in rendered images that contain transparency and refraction, and is particularly notable in scenes with refracted objects that have high-frequency textures. In this work, we propose a neural network architecture to encode critical rendering information, including texture coordinates from refracted rays, and enable reconstruction of high-frequency textures in areas with refraction. Our approach is able to achieve accurate refraction rendering in challenging scenes with a diversity of overlapping transparent objects. Experimental results demonstrate that our method can interactively render high quality refraction effects with global illumination, unlike existing neural rendering approaches. Our code can be found at https://github.com/ziyangz5/CrystalNet
Digital Garment Alteration
(The Eurographics Association and John Wiley & Sons Ltd., 2024) Eggler, Anna Maria; Falque, Raphael; Liu, Mark; Vidal-Calleja, Teresa; Sorkine-Hornung, Olga; Pietroni, Nico; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
Garment alteration is a practical technique to adapt an existing garment to fit a target body shape. Typically executed by skilled tailors, this process involves a series of strategic fabric operations-removing or adding material-to achieve the desired fit on a target body. We propose an innovative approach to automate this process by computing a set of practically feasible modifications that adapt an existing garment to fit a different body shape. We first assess the garment's fit on a reference body; then, we replicate this fit on the target by deriving a set of pattern modifications via a linear program. We compute these alterations by employing an iterative process that alternates between global geometric optimization and physical simulation. Our method utilizes geometry-based simulation of woven fabric's anisotropic behavior, accounts for tailoring details like seam matching, and incorporates elements such as darts or gussets. We validate our technique by producing digital and physical garments, demonstrating practical and achievable alterations.
Gaussian in the Dark: Real-Time View Synthesis From Inconsistent Dark Images Using Gaussian Splatting
(The Eurographics Association and John Wiley & Sons Ltd., 2024) Ye, Sheng; Dong, Zhen-Hui; Hu, Yubin; Wen, Yu-Hui; Liu, Yong-Jin; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
3D Gaussian Splatting has recently emerged as a powerful representation that can synthesize remarkable novel views using consistent multi-view images as input. However, we notice that images captured in dark environments where the scenes are not fully illuminated can exhibit considerable brightness variations and multi-view inconsistency, which poses great challenges to 3D Gaussian Splatting and severely degrades its performance. To tackle this problem, we propose Gaussian-DK. Observing that inconsistencies are mainly caused by camera imaging, we represent a consistent radiance field of the physical world using a set of anisotropic 3D Gaussians, and design a camera response module to compensate for multi-view inconsistencies. We also introduce a step-based gradient scaling strategy to constrain Gaussians near the camera, which turn out to be floaters, from splitting and cloning. Experiments on our proposed benchmark dataset demonstrate that Gaussian-DK produces high-quality renderings without ghosting and floater artifacts and significantly outperforms existing methods. Furthermore, we can also synthesize light-up images by controlling exposure levels that clearly show details in shadow areas.
Density-Aware Diffusion Model for Efficient Image Dehazing
(The Eurographics Association and John Wiley & Sons Ltd., 2024) Zhang, Ling; Bai, Wenxu; Xiao, Chunxia; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
Existing image dehazing methods have made remarkable progress. However, they generally perform poorly on images with dense haze, and often suffer from unsatisfactory results with detail degradation or color distortion. In this paper, we propose a density-aware diffusion model (DADM) for image dehazing. Guided by the haze density, our DADM can handle images with dense haze and complex environments. Specifically, we introduce a density-aware dehazing network (DADNet) in the reverse diffusion process, which can help DADM gradually recover a clear haze-free image from a haze image. To improve the performance of the network, we design a cross-feature density extraction module (CDEModule) to extract the haze density for the image and a density-guided feature fusion block (DFFBlock) to learn the effective contextual features. Furthermore, we introduce an indirect sampling strategy in the test sampling process, which not only suppresses the accumulation of errors but also ensures the stability of the results. Extensive experiments on popular benchmarks validate the superior performance of the proposed method. The code is released in https://github.com/benchacha/DADM.
A Hybrid Parametrization Method for B-Spline Curve Interpolation via Supervised Learning
(The Eurographics Association and John Wiley & Sons Ltd., 2024) Song, Tianyu; Shen, Tong; Ge, Linlin; Feng, Jieqing; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
B-spline curve interpolation is a fundamental algorithm in computer-aided geometric design. Determining suitable parameters based on data points distribution has always been an important issue for high-quality interpolation curves generation. Various parameterization methods have been proposed. However, there is no universally satisfactory method that is applicable to data points with diverse distributions. In this work, a hybrid parametrization method is proposed to overcome the problem. For a given set of data points, a classifier via supervised learning identifies an optimal local parameterization method based on the local geometric distribution of four adjacent data points, and the optimal local parameters are computed using the selected optimal local parameterization method for the four adjacent data points. Then a merging method is employed to calculate global parameters which align closely with the local parameters. Experiments demonstrate that the proposed hybrid parameterization method well adapts the different distributions of data points statistically. The proposed method has a flexible and scalable framework, which can includes current and potential new parameterization methods as its components.
Variable Offsets and Processing of Implicit Forms Toward the Adaptive Synthesis and Analysis of Heterogeneous Conforming Microstructure
(The Eurographics Association and John Wiley & Sons Ltd., 2024) Hong, Q. Youn; Antolin, Pablo; Elber, Gershon; Kim, Myung-Soo; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
The synthesis of porous, lattice, or microstructure geometries has captured the attention of many researchers in recent years. Implicit forms, such as triply periodic minimal surfaces (TPMS) has captured a significant attention, recently, as tiles in lattices, partially because implicit forms have the potential for synthesizing with ease more complex topologies of tiles, compared to parametric forms. In this work, we show how variable offsets of implicit forms could be used in lattice design as well as lattice analysis, while graded wall and edge thicknesses could be fully controlled in the lattice and even vary within a single tile. As a result, (geometrically) heterogeneous lattices could be created and adapted to follow analysis results while maintaining continuity between adjacent tiles. We demonstrate this ability on several 3D models, including TPMS.
CoupNeRF: Property-aware Neural Radiance Fields for Multi-Material Coupled Scenario Reconstruction
(The Eurographics Association and John Wiley & Sons Ltd., 2024) Li, Jin; Gao, Yang; Song, Wenfeng; Li, Yacong; Li, Shuai; Hao, Aimin; Qin, Hong; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
Neural Radiance Fields (NeRFs) have achieved significant recognition for their proficiency in scene reconstruction and rendering by utilizing neural networks to depict intricate volumetric environments. Despite considerable research dedicated to reconstructing physical scenes, rare works succeed in challenging scenarios involving dynamic, multi-material objects. To alleviate, we introduce CoupNeRF, an efficient neural network architecture that is aware of multiple material properties. This architecture combines physically grounded continuum mechanics with NeRF, facilitating the identification of motion systems across a wide range of physical coupling scenarios. We first reconstruct specific-material of objects within 3D physical fields to learn material parameters. Then, we develop a method to model the neighbouring particles, enhancing the learning process specifically in regions where material transitions occur. The effectiveness of CoupNeRF is demonstrated through extensive experiments, showcasing its proficiency in accurately coupling and identifying the behavior of complex physical scenes that span multiple physics domains.
Exploring Fast and Flexible Zero-Shot Low-Light Image/Video Enhancement
(The Eurographics Association and John Wiley & Sons Ltd., 2024) Han, Xianjun; Bao, Taoli; Yang, Hongyu; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
Low-light image/video enhancement is a challenging task when images or video are captured under harsh lighting conditions. Existing methods mostly formulate this task as an image-to-image conversion task via supervised or unsupervised learning. However, such conversion methods require an extremely large amount of data for training, whether paired or unpaired. In addition, these methods are restricted to specific training data, making it difficult for the trained model to enhance other types of images or video. In this paper, we explore a novel, fast and flexible, zero-shot, low-light image or video enhancement framework. Without relying on prior training or relationships among neighboring frames, we are committed to estimating the illumination of the input image/frame by a well-designed network. The proposed zero-shot, low-light image/video enhancement architecture includes illumination estimation and residual correction modules. The network architecture is very concise and does not require any paired or unpaired data during training, which allows low-light enhancement to be performed with several simple iterations. Despite its simplicity, we show that the method is fast and generalizes well to diverse lighting conditions. Many experiments on various images and videos qualitatively and quantitatively demonstrate the advantages of our method over state-of-the-art methods.
Symmetric Piecewise Developable Approximations
(The Eurographics Association and John Wiley & Sons Ltd., 2024) He, Ying; Fang, Qing; Zhang, Zheng; Dai, Tielin; Wu, Kang; Liu, Ligang; Fu, Xiao-Ming; Chen, Renjie; Ritschel, Tobias; Whiting, Emily
We propose a novel method for generating symmetric piecewise developable approximations for shapes in approximately global reflectional or rotational symmetry. Given a shape and its symmetry constraint, the algorithm contains two crucial steps: (i) a symmetric deformation to achieve a nearly developable model and (ii) a symmetric segmentation aided by the deformed shape. The key to the deformation step is the use of the symmetric implicit neural representations of the shape and the deformation field. A new mesh extraction from the implicit function is introduced to construct a strictly symmetric mesh for the subsequent segmentation. The symmetry constraint is carefully integrated into the partition to achieve the symmetric piecewise developable approximation. We demonstrate the effectiveness of our algorithm over various meshes.

Browse

Browsing 43-Issue 7 by Issue Date

Results Per Page

Sort Options