OpenAlex ·
Towards Generalizable Visually Grounded Exploration of Household Devices
- Published
- Source
- OpenAlex
- DOI
- 10.48550/arxiv.2609.00845
- Original
- Read it at the publisher
- Topics
Abstract
Recent advancements in Vision-Language Models (VLMs) have demonstrated impressive capabilities in static visual recognition and high-level semantic reasoning. However, current embodied exploration paradigms still heavily rely on imitation learning from human-annotated trajectories, which severely limits agents' generalization ability. The key bottleneck of realizing general autonomous embodied agents lies in Generalizable Visually Grounded Exploration: the ability to operate novel devices without manuals or specific training by actively grounding abstract world knowledge into fine-grained visual affordances. Yet, existing benchmarks fail to evaluate this capability: they generally rely on explicit documents and annotated trajectories, neglecting the dynamic Hypothesis-Interaction-Refinement process essential for functional device operation. To bridge this gap, we introduce VGEBench, a comprehensive benchmark designed to evaluate the generalizable visually grounded exploration capabilities of VLMs. Unlike static datasets, we construct a Logic-Driven State Machine framework. This framework simulates multi-turn interaction loops, compelling agents to achieve goals by active visual perception and feedback-driven correction. Experimental results demonstrate that existing VLMs face significant challenges in translating semantic knowledge into physical execution and maintaining long-horizon state tracking.
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Artificial Intelligence
Potential-Guided Particle Steering for Negation-Constrained Dexterous Grasping
Global smooth behavior in Kuznetsov and Westervelt type viscous wave equations: A unifying approach covering $W^{1,q}$-small initial data
Facet-0: A Robotic Foundation Model for Contact-Rich Precise Manipulation
The critical exponent for the two-dimensional semilinear wave equation with strong damping
Pole-Zero Geometry, Model Reduction, and Identifiability in Sensory Adaptation
Get the few papers worth your time in Artificial Intelligence every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.