arXiv preprint ·
XARA: A Multimodal Agentic System for eXtended Reality Experiment Generation
- Published
- Source
- arXiv preprint, posted before peer review
- Original
- Read it at arXiv
- Topics
Abstract
Controlled experiments are essential for advancing transportation research, yet designing eXtended Reality (XR) environments for behavioural studies remains technically demanding and inaccessible to many domain researchers. Current workflows require proficiency in game engines such as Unity or Unreal Engine, imposing significant barriers to rapid scenario prototyping and iterative experiment design. This paper presents XARA, a novel system that integrates a dual large language model (LLM) architecture with an immersive XR environment to enable voice driven experiment generation for transportation research. XARA employs a pipeline in which spoken natural language commands are first transcribed and then interpreted into structured commands that manipulate a Unity based XR scene in real time. The system supports object creation, spatial manipulation, animation control, and scene persistence through a modular architecture. By lowering the technical threshold for XR experiment creation, XARA enables transportation researchers to rapidly prototype driving, pedestrian, and urban design scenarios using intuitive voice commands and hand gestures within an immersive 3D environment. This paper describes the system architecture, the voice to command pipeline, and discusses potential applications across behavioural travel experiments, safety studies, human factors research, and policy testing.
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Human-Computer Interaction
MultiPASE: Shape-Aware Comparative 3D Genome Analysis
Kolab: Benchmarking Human-Agent Collaboration in Long-Horizon Workflows
Immersive Focus+Context Techniques to Assist in Interpretation of High Density Forest Point Clouds
OnBodyAI - Why the Next AI Needs a Body
SwitchTube: A Proof-of-Concept System Introducing "Adaptable Commitment Interfaces" as a Tool for Digital Wellbeing
Get the few papers worth your time in Human-Computer Interaction every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.