Nakabayashi, YukiNakamura, FumihikoMasai, KatsutoshiSugimoto, MakiJorge, Joaquim A.Sakata, Nobuchika2025-11-262025-11-262025978-3-03868-278-31727-530Xhttps://doi.org/10.2312/egve.20251347https://diglib.eg.org/handle/10.2312/egve20251347Reconstructing the 3D facial expressions of head-mounted display (HMD) wearers is essential for natural avatar communication in virtual reality (VR). Camera-based methods achieve high fidelity but involve heavy processing and privacy risks, whereas non-imaging sensors are lightweight and privacy-preserving but provide only sparse features. We propose a reconstruction system that learns high-dimensional 3D facial representations from camera images during training, but performs inference using only compact photo-reflective sensors embedded in the HMD. This design integrates the expressiveness of camera-based supervision with the efficiency and privacy of sensor-based operation. Experimental results show that our method accurately reconstructs 3D facial expressions from the sensor data, training with diverse wearing conditions is more effective than collecting more data under a single condition, and accuracy further improves with a dedicated mouth-shape predictor and lightweight personalization using small wearer-specific datasets.Attribution 4.0 International LicenseCCS Concepts: Human-centered computing → Virtual reality; Interaction devicesHuman centered computing → Virtual realityInteraction devices10.2312/egve.2025134710 pages