Conversational Agent for Procedural Building Design in Virtual Reality

Simple item page
dc.contributor.authorBosco, Matteoen_US
dc.contributor.authorKán, Peteren_US
dc.contributor.authorKaufmann, Hannesen_US
dc.contributor.editorHasegawa, Shoichien_US
dc.contributor.editorSakata, Nobuchikaen_US
dc.contributor.editorSundstedt, Veronicaen_US
dc.date.accessioned2024-11-29T06:43:53Z
dc.date.available2024-11-29T06:43:53Z
dc.date.issued2024
dc.description.abstractWith the emergence of large language models (LLMs), conversational agents have gained significant attention across various domains, including virtual reality (VR). This paper investigates the use of conversational agents as an interface for procedural building design in VR. We propose a voice interface that allows a user to control parameters of procedural generation and gain insights about the building construction metrics through natural conversation. The pipeline introduced for the conversational agent involves utilizing LLMs in two separate API calls for natural language understanding and natural language generation. This separation enables the invocation of various actions in procedural generation as well as meaningful agent responses to building-related questions. Furthermore, we conducted a user study to assess our proposed conversational interface in comparison to a traditional graphical user interface (GUI) in a VR architectural design task focused on circular economy. The study scrutinize the user-reported usability, presence, realism, errors, and effectiveness of both interfaces. Results suggest that while the non-embodied conversational agent enhances effectiveness due to its explanatory capabilities, it surprisingly decreases realism compared to the GUI. Overall, the preference between the conversational agent and the GUI varied greatly among participants, highlighting the need for further research into the evolving shift towards speech interaction in VR.en_US
dc.description.sectionheadersNavigation and VRUI
dc.description.seriesinformationICAT-EGVE 2024 - International Conference on Artificial Reality and Telexistence and Eurographics Symposium on Virtual Environments
dc.identifier.doi10.2312/egve.20241378
dc.identifier.isbn978-3-03868-245-5
dc.identifier.issn1727-530X
dc.identifier.pages10 pages
dc.identifier.urihttps://doi.org/10.2312/egve.20241378
dc.identifier.urihttps://diglib.eg.org/handle/10.2312/egve20241378
dc.publisherThe Eurographics Associationen_US
dc.rightsAttribution 4.0 International License
dc.rights.urihttps://creativecommons.org/licenses/by/4.0/
dc.subjectCCS Concepts: Human-centered computing → Virtual reality; Natural language interfaces; Graphical user interfaces; Interactive systems and tools
dc.subjectHuman centered computing → Virtual reality
dc.subjectNatural language interfaces
dc.subjectGraphical user interfaces
dc.subjectInteractive systems and tools
dc.titleConversational Agent for Procedural Building Design in Virtual Realityen_US
Files
Original bundle
Now showing 1 - 3 of 3
Thumbnail Image
Name:
egve20241378.pdf
Size:
1.51 MB
Format:
Adobe Portable Document Format
Download
No Thumbnail Available
Name:
paper1038_mm.mp4
Size:
266 MB
Format:
Video MP4
Download
Thumbnail Image
Name:
paper1038_mm.pdf
Size:
85.84 KB
Format:
Adobe Portable Document Format
Download
Collections
ICAT-EGVE2024