Search Results

Now showing 1 - 10 of 38
  • Item
    On the Beat: Analysing and Evaluating Synchronicity in Dance Performances
    (The Eurographics Association, 2023) Menzel, Malte; Tauscher, Jan-Philipp; Magnor, Marcus; Guthe, Michael; Grosch, Thorsten
    This paper presents a method to analyse and evaluate synchronicity in dance performances automatically. Synchronisation of a dancer's movement and the accompanying music is a vital characteristic of dance performances. We propose a method that fuses computer vision-based extraction of dancers' body pose information and audio beat tracking to examine the alignment of the dance motions with the background music. Specifically, the motion of the dancer is analysed for rhythmic dance movements that are then subsequently correlated to the musical beats of the soundtrack played during the performance. Using a single mobile phone video recording of a dance performance only, our system is easily usable in dance rehearsal contexts. Our method evaluates accuracy for every motion beat of the performance on a timeline giving users detailed insight into their performance. We evaluated the accuracy of our method using a dataset containing 17 video recordings of real world dance performances. Our results closely match assessments by professional dancers, indicating correct analysis by our method.
  • Item
    Immersive Free‐Viewpoint Panorama Rendering from Omnidirectional Stereo Video
    (© 2023 Eurographics ‐ The European Association for Computer Graphics and John Wiley & Sons Ltd., 2023) Mühlhausen, Moritz; Kappel, Moritz; Kassubeck, Marc; Wöhler, Leslie; Grogorick, Steve; Castillo, Susana; Eisemann, Martin; Magnor, Marcus; Hauser, Helwig and Alliez, Pierre
    In this paper, we tackle the challenging problem of rendering real‐world 360° panorama videos that support full 6 degrees‐of‐freedom (DoF) head motion from a prerecorded omnidirectional stereo (ODS) video. In contrast to recent approaches that create novel views for individual panorama frames, we introduce a video‐specific temporally‐consistent multi‐sphere image (MSI) scene representation. Given a conventional ODS video, we first extract information by estimating framewise descriptive feature maps. Then, we optimize the global MSI model using theory from recent research on neural radiance fields. Instead of a continuous scene function, this multi‐sphere image (MSI) representation depicts colour and density information only for a discrete set of concentric spheres. To further improve the temporal consistency of our results, we apply an ancillary refinement step which optimizes the temporal coherency between successive video frames. Direct comparisons to recent baseline approaches show that our global MSI optimization yields superior performance in terms of visual quality. Our code and data will be made publicly available.
  • Item
    Markerless Motion Capture using multiple Color-Depth Sensors
    (The Eurographics Association, 2011) Berger, Kai; Ruhl, Kai; Schroeder, Yannic; Bruemmer, Christian; Scholz, Alexander; Magnor, Marcus; Peter Eisert and Joachim Hornegger and Konrad Polthier
    With the advent of the Microsoft Kinect, renewed focus has been put on monocular depth-based motion capturing. However, this approach is limited in that an actor has to move facing the camera. Due to the active light nature of the sensor, no more than one device has been used for motion capturing so far. In effect, any pose estimation must fail for poses occluded to the depth camera. Our work investigates on reducing or mitigating the detrimental effects of multiple active light emitters, thereby allowing motion capture from all angles. We systematically evaluate the concurrent use of one to four Kinects, including calibration, error measures and analysis, and present a time-multiplexing approach.
  • Item
    PlenopticPoints: Rasterizing Neural Feature Points for High-Quality Novel View Synthesis
    (The Eurographics Association, 2023) Hahlbohm, Florian; Kappel, Moritz; Tauscher, Jan-Philipp; Eisemann, Martin; Magnor, Marcus; Guthe, Michael; Grosch, Thorsten
    This paper presents a point-based, neural rendering approach for complex real-world objects from a set of photographs. Our method is specifically geared towards representing fine detail and reflective surface characteristics at improved quality over current state-of-the-art methods. From the photographs, we create a 3D point model based on optimized neural feature points located on a regular grid. For rendering, we employ view-dependent spherical harmonics shading, differentiable rasterization, and a deep neural rendering network. By combining a point-based approach and novel regularizers, our method is able to accurately represent local detail such as fine geometry and high-frequency texture while at the same time convincingly interpolating unseen viewpoints during inference. Our method achieves about 7 frames per second at 800×800 pixel output resolution on commodity hardware, putting it within reach for real-time rendering applications.
  • Item
    Perception-driven Accelerated Rendering
    (The Eurographics Association and John Wiley & Sons Ltd., 2017) Weier, Martin; Stengel, Michael; Roth, Thorsten; Didyk, Piotr; Eisemann, Elmar; Eisemann, Martin; Grogorick, Steve; Hinkenjann, André; Kruijff, Ernst; Magnor, Marcus; Myszkowski, Karol; Slusallek, Philipp; Victor Ostromoukov and Matthias Zwicker
    Advances in computer graphics enable us to create digital images of astonishing complexity and realism. However, processing resources are still a limiting factor. Hence, many costly but desirable aspects of realism are often not accounted for, including global illumination, accurate depth of field and motion blur, spectral effects, etc. especially in real-time rendering. At the same time, there is a strong trend towards more pixels per display due to larger displays, higher pixel densities or larger fields of view. Further observable trends in current display technology include more bits per pixel (high dynamic range, wider color gamut/fidelity), increasing refresh rates (better motion depiction), and an increasing number of displayed views per pixel (stereo, multi-view, all the way to holographic or lightfield displays). These developments cause significant unsolved technical challenges due to aspects such as limited compute power and bandwidth. Fortunately, the human visual system has certain limitations, which mean that providing the highest possible visual quality is not always necessary. In this report, we present the key research and models that exploit the limitations of perception to tackle visual quality and workload alike. Moreover, we present the open problems and promising future research targeting the question of how we can minimize the effort to compute and display only the necessary pixels while still offering a user full visual experience.
  • Item
    Stereo Inverse Brightness Modulation for Guidance in Dynamic Panorama Videos in Virtual Reality
    (© 2020 Eurographics ‐ The European Association for Computer Graphics and John Wiley & Sons Ltd, 2020) Grogorick, Steve; Tauscher, Jan‐Philipp; Heesen, Nikkel; Castillo, Susana; Magnor, Marcus; Benes, Bedrich and Hauser, Helwig
    The peak of virtual reality offers new exciting possibilities for the creation of media content but also poses new challenges. Some areas of interest might be overlooked because the visual content fills up a large portion of viewers' visual field. Moreover, this content is available in 360° around the viewer, yielding locations completely out of sight, making, for example, recall or storytelling in cinematic Virtual Reality (VR) quite difficult.In this paper, we present an evaluation of Stereo Inverse Brightness Modulation for effective and subtle guidance of participants' attention while navigating dynamic virtual environments. The used technique exploits the binocular rivalry effect from human stereo vision and was previously shown to be effective in static environments. Moreover, we propose an extension of the method for successful guidance towards target locations outside the initial visual field.We conduct three perceptual studies, using 13 distinct panorama videos and two VR systems (a VR head mounted display and a fully immersive dome projection system), to investigate (1) general applicability to dynamic environments, (2) stimulus parameter and VR system influence, and (3) effectiveness of the proposed extension for out‐of‐sight targets. Our results prove the applicability of the method to dynamic environments while maintaining its unobtrusive appearance.
  • Item
    Efficient GPU Based Sampling for Scene-Space Video Processing
    (The Eurographics Association, 2015) Klose, Felix; Wang, Oliver; Bazin, Jean-Charles; Magnor, Marcus; Sorkine-Hornung, Alexander; David Bommes and Tobias Ritschel and Thomas Schultz
    We describe a method to efficiently collect and filter a large set of 2D pixel observations of unstructured 3D points, with applications to scene-space aware video processing. One of the main challenges in scene-space video processing is to achieve reasonable computation time despite the very large volumes of data, often in the order of billions of pixels. The bottleneck is determining a suitable set of candidate samples used to compute each output video pixel color. These samples are observations of the same 3D point, and must be gathered from a large number of candidate pixels, by volumetric 3D queries in scene-space. Our approach takes advantage of the spatial and temporal continuity inherent to video to greatly reduce the candidate set of samples by solving 3D volumetric queries directly on a series of 2D projections, using out-of-core data streaming and an efficient GPU producerconsumer scheme that maximizes hardware utilization by exploiting memory locality. Our system is capable of processing over a trillion pixel samples, enabling various scene-space video processing applications on full HD video output with hundreds of frames and processing times in the order of a few minutes.
  • Item
    Selecting Coherent and Relevant Plots in Large Scatterplot Matrices
    (The Eurographics Association and Blackwell Publishing Ltd., 2012) Lehmann, Dirk J.; Albuquerque, Georgia; Eisemann, Martin; Magnor, Marcus; Theisel, Holger; Holly Rushmeier and Oliver Deussen
    The scatterplot matrix (SPLOM) is a well‐established technique to visually explore high‐dimensional data sets. It is characterized by the number of scatterplots (plots) of which it consists of. Unfortunately, this number quadratically grows with the number of the data set’s dimensions. Thus, an SPLOM scales very poorly. Consequently, the usefulness of SPLOMs is restricted to a small number of dimensions. For this, several approaches already exist to explore such ‘small’ SPLOMs. Those approaches address the scalability problem just indirectly and without solving it. Therefore, we introduce a new greedy approach to manage ‘large’ SPLOMs with more than 100 dimensions. We establish a combined visualization and interaction scheme that produces intuitively interpretable SPLOMs by combining known quality measures, a pre‐process reordering and a perception‐based abstraction. With this scheme, the user can interactively find large amounts of relevant plots in large SPLOMs.The scatterplot matrix (SPLOM) is a well‐established technique to visually explore high‐dimensional data sets. It is characterized by the number of scatterplots (plots) of which it consists of. Unfortunately, this number quadratically grows with the number of the data set's dimensions. Thus, an SPLOM scales very poorly. Consequently, the usefulness of SPLOMs is restricted to a small number of dimensions. For this, several approaches already exist to explore such ‘small’ SPLOMs. Those approaches address the scalability problem just indirectly and without solving it. Therefore, we introduce a new greedy approach to manage ‘large’ SPLOMs with more than 100 dimensions. We establish a combined visualization and interaction scheme that produces intuitively interpretable SPLOMs by combining known quality measures, a pre‐process reordering and a perception‐based abstraction.
  • Item
    A Nonobscuring Eye Tracking Solution for Wide Field-of-View Head-mounted Displays
    (The Eurographics Association, 2014) Stengel, Michael; Grogorick, Steve; Rogge, Lorenz; Magnor, Marcus; Mathias Paulin and Carsten Dachsbacher
    We present a solution for integrating a binocular eye tracker into current state-of-the-art lens-based head-mounted displays (HMDs) without affecting the available field-of-view on the display. Estimating the relative eye gaze of the user opens the door for HMDs to a much wider spectrum of virtual reality applications and games. Further, we present a concept of a low-cost head-mounted display with eye tracking and discuss applications which strongly depend on or benefit from gaze estimation.
  • Item
    Audio Resynthesis on the Dancefloor: A Music Structural Approach
    (The Eurographics Association, 2013) Tauscher, Jan-Philipp; Wenger, Stephan; Magnor, Marcus; Michael Bronstein and Jean Favre and Kai Hormann
    We propose a method for synthesizing a novel soundtrack from an existing musical piece while preserving its structure and continuity from a music theoretical point of view. Existing approaches analyze a musical piece for possible cut points that allow the resynthesis of a novel soundtrack by lining up the source segments according to specified rules but fail to maintain musically correct song progression. Introducing the alignment of rhythmic and harmonic structures during transition point detection, we employ beat tracking as the analysis core component and take the human sound perception into account. Automatic segment rearrangement is improved by employing a novel belief propagation approach that enables user-defined constraints for the output soundtrack, allowing video editors or dance choreographers to tailor a soundtrack to their specific demands.