arXiv preprint ·
Say It or Grab It? Conversational Control of UI Adaptations in Mixed Reality
- Published
- Source
- arXiv preprint, posted before peer review
- Original
- Read it at arXiv
- Topics
Abstract
Mixed reality interfaces must continuously adapt to a user's changing task and environment, yet adaptation policies are fixed in advance, either preset by the system or configured by the user. Natural language offers a way to state changing needs mid-task, but whether users can express interface behavior this way, and what it gains over direct manipulation, is unknown. We first elicited how users verbalize adaptation needs (N=19). We then built ConverseMR, which translates a spoken request into adaptation objectives and constraints and satisfies them continuously as the user's context changes. In a within-subjects study (N=24), conversational and direct control proved complementary: direct manipulation for precision, ConverseMR for reach and expressive breadth, with their combination rated highest. Spoken commands were rarely final and users frequently refined and replaced them upon experiencing the result, characteristic of, what we call, user-led adaptation, where users communicate adaption needs and the system identifies a solution.
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Human-Computer Interaction
GazeAgent: VLM-Guided Implicit Gaze Calibration from Egocentric Multimodal Behavior for AI Glasses
It's Complicated: Factors Influencing the Effectiveness of LLM-Generated Feedback in a Live Programming Course
From Neglect to Resignation: Understanding the Evolution of Mobile Privacy
UnexpectedParrots: Iterating Behavioral Experiment Design with In-Silico Simulations
Simulating Infant Senses with Age-Tuned Networks
Get the few papers worth your time in Human-Computer Interaction every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.