Bridging Frontier Reasoning and Robot Execution: From Autonomous Demonstration Generation to Dense Language Supervision

Abstract

Recent advances in frontier models enable robot manipulation from only a few demonstrations, but high inference latency limits their use for real-time robot control. To bridge this gap, we study two complementary approaches that connect frontier reasoning with low-latency local execution. First, we use a frontier model to autonomously generate demonstrations that supplement human demonstrations for training a fast local policy. We augment its in-context examples with corrective demonstration segments that show how to recover from physical errors, improving generation reliability. Generation time and cost decrease as successful examples accumulate in context, suggesting a path toward more efficient data collection. At deployment, a harness combines frontier-generated instructions with a fast local policy, enabling efficient execution while preserving the frontier model's ability to guide and correct actions. With low-latency execution delegated to the local policy, the bottleneck shifts to its capacity to reliably follow the frontier model's diverse instructions. Our second bridge introduces dense language supervision across three nested granularities---primitive, atomic, and composite---with multi-aspect descriptions at each level. Across long-horizon tasks in RoboCasa 365 and BEHAVIOR-1K, where instructions change as execution progresses, the combined supervision achieves the highest performance under both oracle and frontier-model instructors, demonstrating more reliable instruction following through the policy's language interface. Finally, we evaluate both bridges together on a crossword task that combines semantic planning and manipulation within a fixed time budget. These results support autonomous demonstration generation and dense language supervision as complementary components for connecting frontier reasoning to low-latency local execution.

The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.

More in Robotics

• arXiv preprint · 2 October 2026

  Mind the Refinement Gap: When Safe High-Level Robot Plans Produce Unsafe Executions

  Stabak Das, Priyesh Ranjan, Xiangfang Li and 1 other
• arXiv preprint · 3 October 2026

  Autoware in Construction: Gap Analysis and LiDAR Perception Toward Off-Road Autonomous Driving

  Yu Otsuki, Teja Emmey, Sena Matsushita and 2 others
• arXiv preprint · 2 October 2026

  A Passive AI System for Verifying Physical State on Automated Liquid Handlers

  Junqiong Joanne Qiu, Zeckria Kamrany, Ananya Anand and 7 others
• arXiv preprint · 3 October 2026

  CurveCodec 2: Skeleton-agnostic animation compression with a learned entropy model

  Mingyi Shi, Huancheng Lin, Xuelin Chen and 1 other
• arXiv preprint · 2 October 2026

  DeltaWorld: Physically Consistent Interactive World Simulators via Action-Conditioned Latent Increment Learning

  Boyuan Hou, Xiaoge Cao, Chaofan Zhang and 2 others

All new papers in Robotics