VGFM: Expressive Robot Policies via Dense Value Guidance in Flow Matching

Abstract

Recent robot learning paradigms increasingly rely on large offline datasets of robotic interactions to train control policies. Expressive generative models enable rich and multimodal action representations, expanding the capability of this paradigm for complex robotic control. However, policy improvement with multi-step generative actors remains challenging. In offline reinforcement learning (RL), incorporating value-based objectives along generative trajectories often introduces substantial training complexity, including backpropagation through time (BPTT), auxiliary architectures, or distillation losses. We propose Value-Guided Flow Matching (VGFM), a scalable offline RL framework that enables dense value-guided shaping within a flow-based policy while avoiding BPTT and additional algorithmic overhead. VGFM parameterizes the policy as a conditional flow-matching model in action (x-prediction) space, ensuring that each intermediate flow step produces a valid robot action that can be directly evaluated by a standard offline RL critic. This design allows value guidance to be applied at randomly sampled flow times without differentiating through the entire generative trajectory, while preserving inference-time flexibility by varying the discretization of the underlying flow ODE without retraining. Evaluated on robotic locomotion and manipulation tasks in OGBench, VGFM achieves strong performance across a wide range of tasks under rigorous evaluation protocols. With minimal hyperparameter tuning, these results demonstrate that VGFM provides a simple, scalable, and effective approach for expressive policy learning in long-horizon, goal-oriented robotic control.

The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.

More in Robotics

• arXiv preprint · 13 September 2026

  Novel Ex-vivo Calf Brain Model with Integrated Sub-Skull Force Sensors to Access Simulated Neurosurgical Procedures

  Hamad Binhammad, Matheus Ballestero, Mohammed Babgi and 9 others
• arXiv preprint · 14 September 2026

  Primitive-Informed Sampling-Based MPC for Multi-Fingered Dexterous Manipulation

  Emek Barış Küçüktabak, Karankumar Patel, Jinda Cui and 3 others
• arXiv preprint · 13 September 2026

  ITA-LaCAM: A Complete and Scalable TAPF Solver via Assignment-Aware Configuration-Space Search

  Yimin Tang, Han Zhang, Shao-Hung Chan and 4 others
• arXiv preprint · 14 September 2026

  Real-World Reinforcement Learning with MPC Scaffolding for Dexterous Manipulation

  Emek Barış Küçüktabak, Karankumar Patel, Zhaodong Yang and 3 others
• arXiv preprint · 13 September 2026

  Task-Specified Active Metrological Inspection with Measurement-Steered VLA Manipulation and Deterministic Evidence Gating

  Zhiling Chen, Jingzhan Ge, Ruimin Chen and 3 others

All new papers in Robotics