arXiv preprint ·
Task-State Governance: A Virtual Machine for Verifiable and Recoverable GUI Agent Execution
- Published
- Source
- arXiv preprint, posted before peer review
- Original
- Read it at arXiv
- Topics
Abstract
Graphical user interface (GUI) agents now execute workflows across real applications, but when execution drifts, users must determine what changed, what should have remained untouched, and what can still be recovered. Plan-centric interfaces leave users supervising procedure, but users live with changes to application state. We introduce task-state governance, an interaction paradigm that represents a task as an editable contract linking desired outcomes, facts that must be preserved, and observed application state. We instantiate this paradigm in TaskVM, which projects live state into editable task controls, executes user edits as genuine interface actions, verifies outcomes through independent observation, and marks each action as reversible, compensatable, or irreversible. Across an automated contract benchmark, a controlled user study, and an open live-model exploration, the paradigm strengthened contract compliance and users' recovery control and trust, while execution outcomes were held constant. Task contracts open a design space for governable agent interfaces.
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Human-Computer Interaction
Beyond Initial Creation: Two Decades of Community-based Chart Revisions from Wikimedia Commons
XARA: A Multimodal Agentic System for eXtended Reality Experiment Generation
"The Rose Garden Is a Cemetery": Spatial Dissonance and Divergent Public Experiences of an Indigenous Augmented Reality Tour
Kolab: Benchmarking Human-Agent Collaboration in Long-Horizon Workflows
Measuring and Reducing Unusable Replies of LLM Agents
Get the few papers worth your time in Human-Computer Interaction every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.