Variational Pose Prediction with Dynamic Sample Selection from Sparse Tracking Signals
Loading...
Date
2023
Journal Title
Journal ISSN
Volume Title
Publisher
The Eurographics Association and John Wiley & Sons Ltd.
Abstract
We propose a learning-based approach for full-body pose reconstruction from extremely sparse upper body tracking data, obtained from a virtual reality (VR) device. We leverage a conditional variational autoencoder with gated recurrent units to synthesize plausible and temporally coherent motions from 4-point tracking (head, hands, and waist positions and orientations). To avoid synthesizing implausible poses, we propose a novel sample selection and interpolation strategy along with an anomaly detection algorithm. Specifically, we monitor the quality of our generated poses using the anomaly detection algorithm and smoothly transition to better samples when the quality falls below a statistically defined threshold. Moreover, we demonstrate that our sample selection and interpolation method can be used for other applications, such as target hitting and collision avoidance, where the generated motions should adhere to the constraints of the virtual environment. Our system is lightweight, operates in real-time, and is able to produce temporally coherent and realistic motions.
Description
CCS Concepts: Computing methodologies -> Neural networks; Motion processing; Virtual reality
@article{10.1111:cgf.14767,
journal = {Computer Graphics Forum},
title = {{Variational Pose Prediction with Dynamic Sample Selection from Sparse Tracking Signals}},
author = {Milef, Nicholas and Sueda, Shinjiro and Kalantari, Nima Khademi},
year = {2023},
publisher = {The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {10.1111/cgf.14767}
}