Search Results

Now showing 1 - 10 of 15
  • Item
    Inpainting Normal Maps for Lightstage data
    (The Eurographics Association, 2023) Zuo, Hancheng; Tiddeman, Bernard; Vangorp, Peter; Hunter, David
    This paper presents a new method for inpainting of normal maps using a generative adversarial network (GAN) model. Normal maps can be acquired from a lightstage, and when used for performance capture, there is a risk of areas of the face being obscured by the movement (e.g. by arms, hair or props). Inpainting aims to fill missing areas of an image with plausible data. This work builds on previous work for general image inpainting, using a bow tie-like generator network and a discriminator network, and alternating training of the generator and discriminator. The generator tries to sythesise images that match the ground truth, and that can also fool the discriminator that is classifying real vs processed images. The discriminator is occasionally retrained to improve its performance at identifying the processed images. In addition, our method takes into account the nature of the normal map data, and so requires modification to the loss function. We replace a mean squared error loss with a cosine loss when training the generator. Due to the small amount of available training data available, even when using synthetic datasets, we require significant augmentation, which also needs to take account of the particular nature of the input data. Image flipping and in-plane rotations need to properly flip and rotate the normal vectors. During training, we monitored key performance metrics including average loss, Structural Similarity Index Measure (SSIM), and Peak Signal-to-Noise Ratio (PSNR) of the generator, alongside average loss and accuracy of the discriminator. Our analysis reveals that the proposed model generates high-quality, realistic inpainted normal maps, demonstrating the potential for application to performance capture. The results of this investigation provide a baseline on which future researchers could build with more advanced networks and comparison with inpainting of the source images used to generate the normal maps.
  • Item
    Using The Barnes-Hut Approximation for Fast N-Body Simulations in Computer Graphics
    (The Eurographics Association, 2023) Dravecky, Peter; Stephenson, Ian; Vangorp, Peter; Hunter, David
    Particle systems in CG often encounter performance issues when all the particles rely on mutual influence, producing an O(N2) performance. The Barnes-Hut approximation is used in the field of astrophysics to provide sufficiently accurate results in O(Nlog(N)) time. Here we explore a hardware accelerated implementation of this algorithm, implemented within SideFX Houdini - the commercial tool typically used for particle work in film. We are able to demonstrate a workflow with integrates into the existing artist friendly environment, with performance improved by orders of magnitudes for typically large simulations, and negligible visual change in results.
  • Item
    Classifying User Interface Accessibility for Colourblind Users
    (The Eurographics Association, 2023) Jamil, Amaan; Denes, Gyorgy; Vangorp, Peter; Hunter, David
    Colour vision deficiency (CVD, colourblindness) is the failure or decreased ability to distinguish between certain colours even under normal lighting conditions. There are an estimated 300 million people worldwide with CVD, with approx. 1 in 12 men (8%) and 1 in 200 women (0.5%)
  • Item
    Exploring Language Pedagogy with Virtual Reality and Artificial Intelligence
    (The Eurographics Association, 2023) Michael, Brandon; Aburumman, Nadine; Vangorp, Peter; Hunter, David
    Virtual Reality (VR) is a highly immersive and interactive experience that renders users to be engrossed in a 3D virtual environment. The recent technological advancements with high-resolution headset display, and accurate tracking of six degrees of freedom paired with controllers allow life-like renditions of real-world scenarios as well as fictional scenarios without potential environmental risks. This paper explores the usage of Virtual Reality in education by incorporating current pedagogical approaches into an interactive 3D virtual environment. The focus of this study revolves around language pedagogy, in specific, the tool developed allows teach users fundamental Mandarin Chinese. This educational VR application enables users to practice their reading and writing skills through a calligraphy lesson and engages users in a listening and speaking lesson through natural conversation. To achieve an organic dialogue, phrases spoken by the user in a lesson are validated immediately through an intuitive phrase recognition system developed using machine learning. The developed prototype has undergone testing to ensure its efficacy. An initial investigation into this prototype found that the majority of participants were supportive of this concept and believe that it would improve the engagement of digital education.
  • Item
    An Image-based Model for 3D Shape Quality Measure
    (The Eurographics Association, 2023) Alhamazani, Fahd; Rosin, Paul L.; Lai, Yu-Kun; Vangorp, Peter; Hunter, David
    In light of increased research on 3D shapes and the increased processing capability of GPUs, there has been a significant increase in available 3D applications. In many applications, assessment of perceptual quality of 3D shapes is required. Due to the nature of 3D representation, this quality assessment may take various forms. While it is straightforward to measure geometric distortions directly on the 3D shape geometry, such measures are often inconsistent with human perception of quality. In most cases, human viewers tend to perceive 3D shapes from their 2D renderings. It is therefore plausible to measure shape quality using their 2D renderings. In this paper, we present an image-based quality metric for evaluating 3D shape quality given the original and distorted shapes. To provide a good coverage of 3D geometry from different views, we render each shape from 12 equally spaced views, along with a variety of rendering styles to capture different aspects of visual characteristics. Image-based metrics such as SSIM (Structure Similarity Index Measure) are then used to measure the quality of 3D shapes. Our experiments show that by effectively selecting a suitable combination of rendering styles and building a neural network based model, we achieve significantly better prediction for subjective perceptual quality than existing methods.
  • Item
    Intra-Model Smoothing Using Depth Aware Multi-Sample Anti-Aliasing for Deferred Rendering Pipelines
    (The Eurographics Association, 2023) Magnussen, Birk Martin; Vangorp, Peter; Hunter, David
    Subpixel geometry often causes lighting artifacts. In some cases, post-process anti-aliasing algorithms are not sufficiently able to smooth the resulting image. For forward rendering pipelines, multi-sample anti-aliasing is a powerful tool to avoid such artifacts. However, modern rendering pipelines commonly use deferred shading, which causes issues for multi-sample anti-aliasing. This article proposes a new method of combining a pipeline using deferred shading with multi-sample antialiasing while avoiding common pitfalls. The proposed method achieves this by intelligently resolving the geometry buffers with a custom shader based on the depth of samples. This allows the lighting shader to run unchanged on the geometry buffer on a per-fragment basis without additional performance costs. Furthermore, the proposed method is easy to retrofit to existing engines as no changes are required to either the model rendering shader or the deferred lighting shader. The proposed method is demonstrated and implemented on the example of the open-source game engine FreeSpace Open. It is shown that the proposed method is capable of preventing subpixel geometry artifacts, while also avoiding lighting artifacts from resolving geometry buffers and avoiding the performance overhead of calculating lighting per sample.
  • Item
    AUPE: An Emulator for the ExoMars PanCam Instrument
    (The Eurographics Association, 2023) Ladegaard, Ariel; Gunn, Matt; Miles, Helen C.; Tyler, Laurence; Vangorp, Peter; Hunter, David
    The European Space Agency's ExoMars mission will be the first European-led planetary rover mission and much preparation and rehearsal is required, both for the personnel involved and the data processing pipelines and analysis software. The long instrument development cycle and significant cost associated with flight hardware prohibits their use for extensive field deployment and testing and so emulator systems are required. For this reason an emulator for the PanCam camera system was developed using commercial off-the-shelf components. PanCam's multispectral imaging capabilities will be used to guide the rover to sites of scientific interest, and development of this emulator and the associated data processing techniques are proving invaluable in ensuring the visual-based data products provided to scientists are accurate and that their processing is a transparent and traceable process.
  • Item
    RPS-Net: Indoor Scene Point Cloud Completion using RBF-Point Sparse Convolution
    (The Eurographics Association, 2023) Wang, Tao; Wu, Jing; Ji, Ze; Lai, Yu-Kun; Vangorp, Peter; Hunter, David
    We introduce a novel approach to the completion of 3D scenes, which is a practically important task as captured point clouds of 3D scenes tend to be incomplete due to limited sensor range and occlusion. We address this problem by utilising sparse convolutions, commonly used for recognition tasks, to this content generation task, which can well capture the spatial relationships while ensuring high efficiency, as only samples near the surface need to be processed. Moreover, traditional sparse convolutions only consider grid occupancies, which cannot accurately locate surface points, with unavoidable quantisation errors. Observing that local surface patches have common patterns, we propose to sample a Radial Basis Function (RBF) field within each grid which is then compactly represented using a Point Encoder-Decoder (PED) network. This further provides a compact and effective representation for 3D completion, and the decoded latent feature includes important information of the local area of the point cloud for more accurate, sub-voxel level completion. Extensive experiments demonstrate that our method outperforms state-of-the-art methods by a large margin.
  • Item
    Investigating Deep Learning for Identification of Crabs and Lobsters on Fishing Boats
    (The Eurographics Association, 2023) Iftikhar, Muhammad; Tiddeman, Bernard; Neal, Marie; Hold, Natalie; Neal, Mark; Vangorp, Peter; Hunter, David
    This paper describes a collaboration between marine and computer scientists to improve fisheries data collection. We evaluate deep learning (DL)-based solutions for identifying crabs and lobsters onboard fishing boats. A custom made electronic camera systems onboard the fishing boats captures the video clips. An automated process of frame extraction is adopted to collect images of crabs and lobsters for training and evaluating DL networks. We train Faster R-CNN, Single Shot Detector (SSD), and You Only Look Once (YOLO) with multiple backbones and input sizes. We also evaluate the efficiency of lightweight models for low-power devices equipped on fishing boats and compare the results of MobileNet-based SSD and YOLO-tiny versions. The models trained with higher input sizes result in lower frames per second (FPS) and vice versa. Base models are more accurate but compromise computational and run time cost. Lighter versions are flexible to install with lower mAP than full models. The pre-trained weights for training models on new datasets have a negligible impact on the results. YOLOv4-tiny is a balanced trade-off between accuracy and speed for object detection for low power devices that is the main step of our proposed pipeline for automated recognition and measurement of crabs and lobsters on fishing boats.
  • Item
    Augmenting Anomaly Detection Datasets with Reactive Synthetic Elements
    (The Eurographics Association, 2023) Nikolov, Ivan; Vangorp, Peter; Hunter, David
    Automatic anomaly detection for surveillance purposes has become an integral part of accident prevention and early warning systems. The lack of sufficient real datasets for training and testing such detectors has pushed a lot of research into synthetic data generation. A hybrid approach by combining real images with synthetic elements has been proven to produce the best training results.We aim to extend this hybrid approach by combining the backgrounds and real people captured in datasets with synthetic elements which dynamically react to real pedestrians and create more coherent video sequences. Our pipeline is the first to directly augment synthetic objects like handbags and suitcases to real pedestrians and provides dynamic occlusion between real and synthetic elements in the images. The pipeline can be easily used to produce a continuous stream of randomized augmented normal and abnormal data for training and testing. As a basis for our augmented images, we use one of the most widely used classical datasets for anomaly detection - the UCSD dataset. We show that the synthetic data produced by our proposed pipeline can be used to make the dataset harder for state-of-the-art models, by introducing more varied and challenging anomalies. We also demonstrate that the additional synthetic normal data can boost the performance of some models. Our solution can be easily extended with additional 3D models, animations, and anomaly scenarios.