arXiv preprint ·
Same Goal, Different Words: Spatial Instruction Sensitivity in World-Action Models
- Published
- Source
- arXiv preprint, posted before peer review
- Original
- Read it at arXiv
- Topics
Abstract
World-action models should preserve the intended manipulation across equivalent descriptions of a spatial goal. We evaluate this consistency in 864 rollouts from three models across four RoboLab scenes, holding initial states and task goals fixed. Describing relations through the reference object rather than the object-to-move lowers the rate of reaching a stable placement by 12.2 percentage points on average. The effect of instruction rephrasing varies across spatial relations, so aggregate scores can hide sensitivity to wording. Standalone tests of the policies' language understanding identify relation reversals and show that adding images can disrupt otherwise correct text-only interpretations. Rollout analysis also distinguishes spatial placement from instruction following: moving the wrong object can satisfy the requested relation, and rates of stable placement fall from 43.2% at any time to 21.8% at episode end. These findings motivate a more thorough evaluation of world-action models' steerability, language understanding, and grounding. Evaluations should test consistency across equivalent instructions, verify that the intended object moved, and check whether the requested arrangement persists to the end of the rollout.
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Robotics
PathLift: Depth-Conditioned Metric Lifting for Visual-Navigation Priors
An Optimal Algorithm for Changing from Longitudinal to Latitudinal Formation of Autonomous Aircraft Squadrons
Constrained Differentiable Policy Learning for Safe Quadrotor Navigation
YUBI-Real2Sim: 3DGS Scene Verification for Reliable Policy Evaluation
Directional Pullback Regularization for Long-Horizon Rollout of Latent World Models in Autonomous Driving
Get the few papers worth your time in Robotics every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.