arXiv preprint ·
Bringing physics-based controller policies to virtual reality
- Published
- Source
- arXiv preprint, posted before peer review
- Original
- Read it at arXiv
- Topics
Abstract
Nowadays, it is easy to set up any element of a VR experience so users can interact with it physically, except for interactive VR characters. Can we extend this to animated VR characters? In this work, we studied whether physics-based character animation (PBCA) controllers can produce high-quality movement in VR-compatible environments (Unity3D, WebXR). We evaluated the quality of the results, as well as the losses at each stage of the processing pipeline: data ingest, training and inference. For data ingest, we computed accuracy metrics related to the retargeting method used. For training, we computed accuracy metrics related to different training policies and different training environments. We developed new JAX implementations of state-of-the-art PBCA controllers (DeepMimic, AMP and ADD) which can be trained in MuJoCo-MJX in minutes, 50 times faster than in a traditional CPU-based training environment. The benefit of these particular implementations is that the resulting control policies can be readily transferred to Unity3D and support cross-platform compilation, including Android, something critical for use in VR production. For inference, we tested these controller policies in Unity3D and WebXR. Our results suggest that DeepMimic and ADD trained in MuJoCo-MJX generate controller policies with better animation fidelity than other methods. To simplify the comparison between controller policies, and different reference animations, we implemented an animation fidelity index (afi). Its results are analogous to other more common metrics, but afi is more intuitive to interpret and thus simplifies the comparison of training methods across the successive stages of the processing pipeline.
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Human-Computer Interaction
When Better Opportunities Backfire: Service Worker Responses to Parallel AI Deployment in Customer Acquisition
Relational Complexity: Making Sense of Social Companion AI Use
When Care Expands and Control Declines: Understanding Digital Privacy in End-of-Life Care
Bias and Reliability in Learning-Style Inference from Student--AI Dialogues: An Evaluation of GPT-4o and Gemini 2.0 Flash
Cling: Supporting Cross-Page Coordination with Human-Directed, In-Situ, and Small-Step Automation
Get the few papers worth your time in Human-Computer Interaction every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.