UDAV: Uncertainty-Driven Adaptive VLM Waypoint Planner

Abstract

Vision-language models (VLMs) can generate routes directly from aerial imagery for off-road navigation, but their predictions provide no indication of reliability. We present UDAV, an Uncertainty-Driven Adaptive VLM Waypoint Planner for UAV-guided UGV navigation. UDAV draws multiple stochastic trajectory predictions, selects their medoid as a self-consistent nominal route, and estimates predictive uncertainty from their spatial dispersion. When the maximum uncertainty across interior waypoints exceeds a threshold, UDAV invokes a reconsideration stage; otherwise, it returns the medoid directly. We evaluate UDAV on 400 held-out trajectory queries from two UAV flights. Stochastic medoid selection reduces the mean average displacement error (ADE) from 147.4 pixels for a deterministic prediction to 115.9 pixels. The complete planner achieves a mean ADE of 110.4 pixels, a 25.1% reduction relative to deterministic planning, while producing valid trajectories for all queries. UDAV also yields the lowest 90th- and 95th-percentile errors among all evaluated configurations, including a higher-budget K=10 consensus baseline. Relative to the K=5 medoid, UDAV reduces these errors from 225.3 and 326.0 pixels to 199.0 and 290.8 pixels, respectively. These results demonstrate that stochastic VLM predictions provide both a stronger nominal route and an actionable uncertainty signal for selectively mitigating large planning errors.

The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.

More in Robotics

• arXiv preprint · 14 September 2026

  Primitive-Informed Sampling-Based MPC for Multi-Fingered Dexterous Manipulation

  Emek Barış Küçüktabak, Karankumar Patel, Jinda Cui and 3 others
• arXiv preprint · 15 September 2026

  Dense to MoE Adaptation for Compact Vision Language Action Policies

  Muchun Niu, Shuang Chen, Yuzhou Wu and 1 other
• arXiv preprint · 14 September 2026

  Real-World Reinforcement Learning with MPC Scaffolding for Dexterous Manipulation

  Emek Barış Küçüktabak, Karankumar Patel, Zhaodong Yang and 3 others
• arXiv preprint · 15 September 2026

  UniDex-ViTac: Learning Unified Visuo-Tactile Dexterous Manipulation Policy from Human Video Data

  Hyesung Lee, Si-Hwan Heo and Sungwook Yang
• arXiv preprint · 14 September 2026

  HydroMap: Probabilistic Water Surface Elevation Mapping for Semantic Scene Representation in Inland Waterways

  Zhongbi Luo, Yunjia Wang, Herman Bruyninckx and 1 other

All new papers in Robotics