ICAT-EGVE2025

Permanent URI for this collection

https://diglib.eg.org/handle/10.2312/3607269

Browse

Now showing 1 - 18 of 18

AR-Eye: A Custom, Low-Cost Eye Tracker for Mental Fatigue Detection with Pattern-Based Machine Learning on Augmented Reality Headsets
(The Eurographics Association, 2025) Stavroulakis, Alexios; Roumeliotis, Michail; Safranoglou, Ioannis; Ramiotis, George; Mania, Katerina; Jorge, Joaquim A.; Sakata, Nobuchika
We present a low-cost, custom-built eye tracking system designed as a modular add-on for augmented reality (AR) headsets that either lack native eye tracking capabilities or provide insufficient ocular data for cognitive monitoring. Our system leverages infrared-enabled cameras and machine learning techniques to extract oculometric features-pupil diameter, blink dynamics, and saccadic velocity-in real time, enabling reliable detection of user mental fatigue. Unlike commercial high-cost solutions or restricted APIs of built-in AR trackers, the proposed device provides direct access to raw eye metrics under diverse conditions. We implement an unsupervised clustering approach combined with supervised classifiers to estimate fatigue levels on a persecond basis, integrating both self-reports and objective eye tracking data. Results are streamed into the AR environment, where individual and team-wide fatigue states are visualized through intuitive holographic overlays. This collaborative monitoring framework enables users to track both their own and others' cognitive states, supporting adaptive interventions in shared AR tasks. The system demonstrates the feasibility of affordable, portable, and extensible ocular-based fatigue detection for enhancing performance, safety, and well-being in group-oriented AR applications.
Balancing Speed and Visual Fidelity of Dynamic Point Cloud Rendering in VR
(The Eurographics Association, 2025) Muehlenbrock, Andre; Weller, Rene; Zachmann, Gabriel; Jorge, Joaquim A.; Sakata, Nobuchika
Efficient rendering of dynamic point clouds from multiple RGB-D cameras is essential for a wide range of VR/AR applications. In this work, we introduce and leverage two key parameters in a mesh-based rendering approach and conduct a systematic study of their impact on the trade-off between rendering speed and perceptual quality. We show that both parameters enable substantial performance improvements while causing only negligible visual degradation. Across four GPU generations and multiple deployment scenarios, continuous dynamic point clouds from seven Microsoft Azure Kinects can achieve binocular rendering at triple-digit frame rates, even on mid-range GPUs. Our results provide practical guidelines for balancing visual fidelity and efficiency in real-time VR point cloud rendering, demonstrating that mesh-based approaches are a scalable and versatile solution for applications ranging from consumer headsets to large-scale projection systems.
Beyond Traditional Portals: Redirected Portals with Destination Preview for Walking in VR
(The Eurographics Association, 2025) Ribeiro, André Tomás; Rebelo, Ana Rita; Nóbrega, Rui; Jorge, Joaquim A.; Sakata, Nobuchika
Natural locomotion in Virtual Reality (VR) is often constrained by the size of the physical tracking space. Portals - virtual connections between distant locations - offer a promising solution to support natural walking in limited physical areas. However, traditional portal implementations often introduce usability challenges, such as requiring large clearances from walls to prevent physical collisions, or creating unnatural transitions between spaces. In this paper, we explore four portal designs for enabling natural walking in impossible spaces: (1) Traditional Portal with a spatial preview and offset from walls; (2) Movable Portal that initially appears on a wall and can be repositioned by the user; (3) Interactive Door: a door-shaped portal embedded in the wall that can be opened interactively; (4) Revolving Door: a portal offering a continuous transition between rooms through a circular motion. These designs emerged from initial user trials that revealed the desire for more flexible and natural transitions between rooms. We conducted a user study (N=31) to evaluate these four portal types, focusing on usability, spatial understanding, immersion, and perceived naturalness. Results suggest that the alternative designs addressed the portal limitations successfully and helped convey a continuous sense of space, with each portal design revealing their strong suits. Movable Portals were commended for their sense of control, Interactive Doors for their naturalness and ease-of-use and Revolving Doors for the best feel of a continuous space, with Traditional Portals being the most preferred for their high ease-of-use.
Developing a Controlled Experimental Space in VRChat: Comparing Embodiment in VRChat Users and Non-VRChat Users
(The Eurographics Association, 2025) Kondo, Ryota; Sakurada, Kuniharu; Oyanagi, Akemi; Tanikawa, Tomohiro; Hirose, Michitaka; Jorge, Joaquim A.; Sakata, Nobuchika
In virtual reality, it is possible for users to feel as if the avatar were their own innate body, a phenomenon known as the sense of embodiment. In conventional laboratory experiments, the movements and observations that induce avatar embodiment have typically been limited to a short duration of about five minutes. In contrast, in social VR, users are considered to naturally engage in prolonged movements and observations, and it is possible that the sense of embodiment is strengthened through long-term play. In this study, we constructed an experimental space in VRChat that enables the controlled manipulation and measurement of the sense of embodiment, and we compared VRChat users with non-users. The results showed that there was no significant difference in the sense of embodiment depending on whether or not participants had prior VRChat experience. On the other hand, among VRChat users, it was revealed that the component of embodiment known as self-location became stronger as playtime increased. In addition, our findings confirmed that the control and measurement of embodiment can be conducted in VRChat in a manner comparable to laboratory experiments. By utilizing the experimental space developed in this study, it will be possible in the future to conduct various experiments that investigate VRChat users in the extended context of the environments in which they live.
Evaluating the Effect of Multimodal Scenario Cues in an LLM-Supported Auditory VR Design System for Exposure Therapy
(The Eurographics Association, 2025) Yamauchi, Yuta; Tsuji, Yuta; Ino, Keiko; Sakaguchi, Masanori; Zempo, Keiichi; Jorge, Joaquim A.; Sakata, Nobuchika
Post-Traumatic Stress Disorder (PTSD) is a prevalent disorder triggered by life-threatening trauma, and exposure therapy, which involves confronting traumatic stimuli, has been proven to be highly effective for treating PTSD. Some methods have been proposed that present patients' traumatic situations using spatial sound (Auditory VR) to provide a sense of realism during therapy (Auditory VR Exposure Therapy). However, the Auditory VR used in therapy need to be tailored for each patient, and conventionally they have been manually produced by third parties, resulting in long delays before Auditory VR exposure therapy can begin. In the previous work, they developed a system that enables the creation of auditory stimuli through text-only interaction, allowing clinicians and patients to generate sounds without the involvement of third parties. However, creating Auditory VR solely through natural language interaction proved challenging, leading to issues in usability and sound quality. In this study, while maintaining the basic approach of text-based Auditory VR generation, we developed a system that enables multimodal interaction by combining text with auditory presentation of scenarios and visual presentation of spatial designs. An evaluation experiment with participants with backgrounds in medicine or healthcare demonstrated that, compared to the previous system, the system improved usability and, based on subjective evaluations, achieved significantly higher overall sound quality ratings.
Exploration of Delays and Tonality for LLM-powered Game Narrators
(The Eurographics Association, 2025) Lind, Arlonsompoon P.; Lind, Jonas B.; Sørensen, Mads W.; Jacobsen, Rasmus V.; Whitehead, Sebastian M.; Nikolov, Ivan; Jorge, Joaquim A.; Sakata, Nobuchika
As Large Language Models (LLMs) are becoming more widely spread as part of games and interactive media, more research is needed in introducing them in areas where they can provide deeper immersion, interactivity, and customization. The role of LLMs as storytellers and narrators has the potential for just that however, not enough research has been done focusing on how users react to real-time storytelling by AI. In this research, we test several factors associated with integrating LLM narrators in an interactive experience. Namely, acceptable narration delay for intentional and unintentional user-triggered interactions, for exploration of an environment, and in connection with the importance of the objects in the environment, and finally, how these factors are connected to user perception of LLM narration tonality in a VR historical setting. As part of the results, we provide recommendations for acceptable delays in narration, connecting narration to objects, and the choice of tonality of the LLM.
Exploring the Effects of Spatially Distributed Cheek Pressure Stimulation on Virtual Experiences
(The Eurographics Association, 2025) Kimura, Ryu; Nakamura, Fumihiko; Kimura, Asako; Shibata, Fumihisa; Jorge, Joaquim A.; Sakata, Nobuchika
In Virtual Reality (VR), haptic stimulation is promising approach to enhancing virtual experiences. However, facial haptics remain underexplored, particularly the cheeks, which have the potential to improve presence and to foster affective interaction. This study explores the effects of spatially distributed cheek pressure stimulation on virtual experiences. We developed a facial stimulation system by attaching six servo-driven linear actuators to a head-mounted display, enabling the application of pressure to multiple points on the cheek. Two user studies were conducted under three conditions: visual-only, single-point stimulation, and multi-point stimulation. The first study examined the effects of multi-point cheek pressure stimulation on presence and emotional responses when interacting with non-human virtual entities. The second study assessed its impact on presence, emotional responses, and social presence during interactions with human-like avatars. Results showed that cheek pressure stimulation significantly enhanced presence and social presence, and elicited higher emotional arousal. Furthermore, temporal variations in cheek pressure effectively improved valence in playful interaction with an avatar. Comparisons between singlepoint and multi-point stimulation revealed only minor differences, indicating limited additional benefits of multiple pressure points. However, participants' self-reports suggest that applying pressure to multiple points on the cheek is more likely to increase the perceived overall force intensity, rather than producing a sensation of distinct stimulation points.
Exploring the Use of Auditory Feedback as a Guide for 3D Drawing in Extended Reality
(The Eurographics Association, 2025) Gruy, Esther; Berthaut, Florent; Jorge, Joaquim A.; Sakata, Nobuchika
3D drawing (or sketching) in Extended Reality (XR) is more difficult to master than 2D drawing. It is performed in mid-air and does not benefit from a physical surface, has more degrees of freedom, and is often dependent on Head-Mounted Displays that contain inaccurate stereoscopic rendering (due to the vergence-accommodation conflict). Many approaches were explored in the scientific literature in order to compensate for these issues, such as visual guides, the use of haptic feedback, or beautifying techniques. However, very few focus on the use of auditory feedback as a possible guide for increasing spatial awareness, and thus accuracy, while drawing inside a 3D space, although sound has proven to be a useful additional feedback in several 3D interactive contexts. In this paper, we explored several auditory feedback (based on the depth of the controller and the proximity to other strokes in the canvas) to try and see if it improves the accuracy of drawings in XR. To do so, we conducted an experiment with 21 participants, mostly novices in drawing. The results show that the use of the proposed auditory feedback do not have an influence on the accuracy of the drawings, but do have an influence on the participants' perceived accuracy and the confidence in their ability to perform the task.
Facial Expression Reconstruction with Photo-Reflective Sensors Embedded in a Head-Mounted Display
(The Eurographics Association, 2025) Nakabayashi, Yuki; Nakamura, Fumihiko; Masai, Katsutoshi; Sugimoto, Maki; Jorge, Joaquim A.; Sakata, Nobuchika
Reconstructing the 3D facial expressions of head-mounted display (HMD) wearers is essential for natural avatar communication in virtual reality (VR). Camera-based methods achieve high fidelity but involve heavy processing and privacy risks, whereas non-imaging sensors are lightweight and privacy-preserving but provide only sparse features. We propose a reconstruction system that learns high-dimensional 3D facial representations from camera images during training, but performs inference using only compact photo-reflective sensors embedded in the HMD. This design integrates the expressiveness of camera-based supervision with the efficiency and privacy of sensor-based operation. Experimental results show that our method accurately reconstructs 3D facial expressions from the sensor data, training with diverse wearing conditions is more effective than collecting more data under a single condition, and accuracy further improves with a dedicated mouth-shape predictor and lightweight personalization using small wearer-specific datasets.
ICAT-EGVE 2025 - Frontmatter
(The Eurographics Association, 2025) Jorge, Joaquim A.; Sakata, Nobuchika; Jorge, Joaquim A.; Sakata, Nobuchika
Impact of Visual, Auditory, and Mixed Interfaces on Human-Robot Collaboration in Multi-Robot Environments
(The Eurographics Association, 2025) Nagahara, Takumi; Techasarntikul, Nattaon; Ohsita, Yuichi; Shimonishi, Hideyuki; Jorge, Joaquim A.; Sakata, Nobuchika
In the field of Human Robot Collaboration (HRC) research, many studies have explored the use of visual and/or auditory cues as robot caution interfaces. However, many of these studies have focused on interfaces, such as displays of a single robot's future position or hazardous areas, without validating them in complex environments where multiple robots operate simultaneously and users need to perceive and respond to multiple robots at once. An increase in the number of robots can exceed human cognitive limits, potentially leading to a decrease in safety and operational efficiency. To achieve safe and work efficient HRC in environments with multiple robots, we proposed a design for auditory and visual augmented reality interfaces to help workers be aware of multiple robots. We evaluated both single-modal and multi-modal interfaces under varying numbers of robots in the environment to explore how user perception and safety are affected. We conducted a comparative evaluation using multiple metrics, including the number of collisions, the closest distance to a robot, interface response time, task completion time, and subjective measures. Although multi-modal interfaces can reduce the average number of collisions by approximately 19%- 49% compared to single-modal interfaces, and generally outperform them, their relative advantage diminished as the number of robots increased. This may be attributed to the physical limitations of the environment, where avoiding multiple robots simultaneously becomes inherently difficult, thereby reducing the impact of interface design on user performance.
Influence of Non-Isomorphic Interactions on Users' Agency with a Dissimilar Avatar in Virtual Reality
(The Eurographics Association, 2025) Brument, Hugo; Podkosova, Iana; Ratschiller, Gabriel; Vailland, Damien; Kaufmann, Hannes; Jorge, Joaquim A.; Sakata, Nobuchika
The user's perception in Virtual Reality (VR) with avatars is strongly influenced by the Sense of Embodiment (SoE) and Sense of Agency (SoA). However, traditional VR interfaces often rely on human-centric interactions (e.g., hand or raycast) that may not translate well to dissimilar avatars that differ from human anatomy. This paper describes the design and evaluation of selection and navigation metaphors in VR for a dissimilar avatar. We introduce StretchIK, an adaptation of the Forward Backward Reaching Inverse Kinematics that works with non-isomorphic interaction metaphors. We conducted two user studies to investigate the influence of the interaction metaphors on users' SoE and SoA. In the first study, we compared two selection metaphors (Go-go versus Head-Selection), where participants had to grab virtual cabbages. In the second study, we compared two navigation metaphors (3D steering versus 3D Leaning), where participants had to travel through virtual rings. The results showed that users had a good SoE in the dissimilar avatar for interacting. However, the interaction metaphors affected users' performance, SoA, where Go-Go outperformed Head-Selection, and 3D steering outperformed 3D Leaning. Our results could improve interaction with dissimilar avatars, eliciting potential future work regarding shared control of dissimilar avatars in VR.
Modeling Audience NPC's Diversity for Enhanced Illusionary Sociality in Collective VR Concert Experiences
(The Eurographics Association, 2025) Ishinuki, Takehiro; Fujishiro, Issei; Jorge, Joaquim A.; Sakata, Nobuchika
Virtual reality (VR) live performances offer a novel style of experience through the deployment of three-dimensional cartoonlike performers and through spatiotemporally asynchronous participation. However, such experiences often lack the presence and atmospheric richness of physical live concerts. To address this gap, prior research has proposed audience non-player characters (NPCs) that emulate the collective characteristics of real-world audiences to enhance viewer engagement. In this study, we extend existing approaches by incorporating individual motion diversity to synthesize audience NPCs that form a heterogeneous and lifelike crowd. In addition to collective characteristics, such as music-driven motion changes and gradual synchronization, our modeling approach introduces agent-level diversity, even within the same motion categories. To assess the psychological impact of the proposed method, we developed a VR concert scenario featuring approximately 1,500 audience NPCs and conducted an empirical evaluation of the user experience. The findings indicate that enhanced motion heterogeneity can increase a viewer's sense of presence, engagement, perceived unity, and co-existence in a VR concert environment.
Preliminary Demonstration of Aerial 2D Projection via CT-Based Ray Superposition in Fog
(The Eurographics Association, 2025) Nishizawa, Masahiro; Yamada, Muso; Sakai, Goki; Nakata, Takayuki; Jorge, Joaquim A.; Sakata, Nobuchika
Volumetric and aerial displays have been pursued as a means to present three-dimensional images viewable from any direction without head-mounted devices or glasses. However, existing approaches often suffer from limitations such as restricted viewing zones, narrow display volumes, or dependence on large screens or mechanically moving components. To address these challenges, we propose a new approach based on computed tomography (CT), which applies CT principles to projector rays visualized in fog. In this method, images are reconstructed in mid-air by superimposing rays projected from multiple angles, eliminating the need for physical projection surfaces. In this paper, we report a preliminary demonstration of this approach. We built a prototype with ten scanning laser projectors arranged in a circular configuration and tested it in a fog-filled environment. The system successfully produced faint yet recognizable two-dimensional aerial images consistent with the intended shapes. This experiment confirms the feasibility of ray-superimposed back-projection in real space, not just in simulation. This preliminary result represents an initial step toward scalable, screenless, and multi-viewable volumetric displays.
Projecting BRDF Materials using Polarimetric Normal Estimation
(The Eurographics Association, 2025) Matsuno, Sota; Amano, Toshiyuki; Jorge, Joaquim A.; Sakata, Nobuchika
This study proposes a method based on Spatial Augmented Reality (SAR) technology to transform the perceived material of an object's surface by projecting imagery rendered from measured BRDF data. The core of this method lies in the non-contact estimation of the object's three-dimensional surface shape from images captured at four different polarization angles. Specifically, Stokes parameters are calculated from four polarized images, and a normal vector is derived for each pixel based on the resulting Angle of Linear Polarization (AoLP) and Degree of Linear Polarization (DoLP). Next, rendering is performed using the estimated normal map and a measured BRDF database. The resulting appearance of the target material is then projected onto the object via a projector-camera system. Experiments demonstrate the ability to change the material appearance of certain objects, such as plaster statues and plastic products. The significance of this research lies in demonstrating a new framework for presenting material appearance by combining shape estimation using polarization with physically based rendering.
ProTrans: Projecting In-Place Translations for Printed Text
(The Eurographics Association, 2025) Grünwald, Dustin; Imoto, Yusuke; Taninaka, Isabella Mika; Sato, Kosuke; Iwai, Daisuke; Jorge, Joaquim A.; Sakata, Nobuchika
Reading unknown words in non-native languages can hinder comprehension or slow reading by requiring dictionary consultation. Existing solutions for faster lookup only work digitally or, in the case of printed text, require a separate display. To enable a more seamless reading experience in the latter scenario, we present ProTrans (Projected Translation), a projector-camera system that detects words users point to and projects their translations onto nearby surfaces. We compare three projection targets-paper, finger, and hand-and gather initial user feedback. Results indicate that projecting onto the back of the hand balances legibility and viewing comfort, supporting the feasibility of skin-based projection for translation tasks.
SCENTAUR - An Encountered-Type Olfactory Display for Large-Scale Smell Interaction in Virtual Reality
(The Eurographics Association, 2025) Brument, Hugo; Pace, Francesco De; Götz, Christoph; Kaufmann, Hannes; Jorge, Joaquim A.; Sakata, Nobuchika
Recent years have shown a greater interest in incorporating the sense of smell into Virtual Reality (VR) through olfactory displays designed to generate and deliver scent stimuli in virtual environments. Current designs often rely on stationary devices (preventing large-scale or mobile scent delivery in the workspace) or wearable devices (that may be cumbersome to users), missing opportunities to consider the use of robots to deliver smell in a large-scale workspace to improve immersive VR. This paper introduces SCENTAUR, an Encountered-Type Olfactory Display (ETOD) featuring a stationary smell display attached to a mobile robot that can navigate and display smell in a large workspace to allow users to experience smell in VR while walking in a virtual environment without wearing additional equipment. We evaluated the usability of SCENTAUR in providing a perceptible large-scale smell interaction in VR through a user study. The findings highlight that SCENTAUR can be a plausible olfactory display that interacts with a walk-based VR setup. As a part of our contribution, a proof of concept was designed to illustrate our approach. We discuss our results based on the current state-of-the-art smell displays in VR and the design implications of ETODs where SCENTAUR could improve spatial user(s) interaction.
Trust and Safety in Autonomous Vehicles: Evaluating Contextual Visualizations for Highlighting, Prediction, and Anchoring
(The Eurographics Association, 2025) Tran, Thi Thanh Hoa; Peillard, Etienne; Walsh, James; Moreau, Guillaume; Thomas, Bruce; Jorge, Joaquim A.; Sakata, Nobuchika
For autonomous vehicles (AVs) to be widely accepted, users must not only feel safe but also understand how the vehicle perceives and responds to its environment. Augmented Reality (AR) enables real-time, intuitive communication of such information, helping foster trust and enhance situation awareness (SA). This paper presents the results of three online user studies that investigate the design of different AR visualization strategies in simulated AV environments. Although the studies used prerecorded videos, they were designed to simulate ecologically realistic driving scenarios. Study 1 evaluates six types of highlight visualizations (bounding box, spotlight, point arrow, zoom, semantic segmentation, and baseline) across five driving scenarios varying in complexity and visibility. The results show that highlight effectiveness is scenario-dependent, with bounding boxes and spotlights being more effective in occluded or ambiguous conditions. Study 2 explores predictive visualizations, comparing single vs. multiple predicted paths and goals to communicate future trajectories. Findings indicate that single-path predictions are most effective for enhancing trust and safety, while multi-goal visualizations are perceived as less clear and less helpful. Study 3 examines the impact of spatial anchoring in AR by comparing screen-fixed and world-fixed presentations of timeto- contact information. Results demonstrate that world-fixed visualizations significantly improve trust, perceived safety, and object detectability compared to screen-fixed displays. Together, these studies provide key insights into when, what, and how AR visualizations should be presented in AVs to effectively support passenger understanding. The findings inform the design of adaptive AR interfaces that tailor visual feedback based on scenario complexity, uncertainty, and environmental context.

Browse

Browsing ICAT-EGVE2025 by Title

Results Per Page

Sort Options