Revealing and Bridging the Physical Executability Gap in Vision-Language Navigation

Abstract

Continuous vision-and-language navigation (VLN) is typically evaluated by goal-reaching success, assuming it reflects an agent's ability to execute language-grounded behavior. We show that this assumption can fail: permissive contact models in current simulators may mask physically infeasible actions, creating a physical executability gap between semantic competence and embodied realization. To bridge this gap, we introduce Visual-Semantic Traversability Alignment (VeSTA), a framework that aligns VLN policies with physically executable futures. We define visual-semantic traversability as the ability to predict instruction-conditioned trajectories that are semantically valid, geometrically feasible, and physically robust. VeSTA addresses this misalignment through two complementary mechanisms: improving executable support in trajectory generation and risk-aware selection among multimodal futures. Specifically, Trajectory Relative Group Policy Optimization uses strict-contact feedback to reshape the candidate distribution toward executable regions, while the Traversability Risk Critic learns a risk-adjusted scalar utility for candidate-wise future selection. Experiments on R2R and RxR reveal substantial dependence of existing VLN agents on simulator contact semantics. Under strict-contact evaluation, VeSTA substantially reduces collision-induced failures while preserving or improving goal-reaching performance, and controlled real-world robot experiments provide evidence that the learned preference transfers beyond simulation.

The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.

More in Robotics

• arXiv preprint · 19 October 2026

  Vector-Quantized World Model: Latent Feedback Motion Planning for Robot Manipulation

  Xuezhou Xu, Chung-Pang Wang, Zhenzhen Li and 1 other
• arXiv preprint · 20 October 2026

  Real-Time Disturbance-Aware Koopman Model Predictive Control for Quadrotors in Spatially Varying Winds

  Stella Kombo, Masih Haseli, Alejandro A. Stefan-Zavala and 3 others
• arXiv preprint · 19 October 2026

  Humanoid SRL: Augmenting Humanoids with Supernumerary Limbs for Coordinated Loco-Manipulation

  Fukang Liu, Feiyang Wu, Paul Barsa and 12 others
• arXiv preprint · 20 October 2026

  Towards Autonomous Bricklaying Using Boston Dynamics Spot

  Karl-Johan Sorensen, Nico vom Hofe and Frank H. P. Fitzek
• arXiv preprint · 19 October 2026

  Where Do the Gaussians Go? Foreground Attraction for Camera-Based Weakly Supervised 3D Occupancy

  Jeagon Song and Jhonghyun An

All new papers in Robotics