VLaRL: Augmenting Vision-Language-Action Models with Simulation-Trained Latent-Conditioned Residual RL

Abstract

Vision-language-action (VLA) models provide broad, instruction-conditioned manipulation behaviors, but their physical execution can remain imprecise during contact-rich interaction. Residual reinforcement learning (RL) can correct such errors while keeping the VLA frozen, but real-robot RL is costly and safety-critical. We propose VLA Latent-Conditioned RL (VLaRL), which enables residual RL for frozen VLAs to be trained in simulation and deployed on real robots without real-world RL or online adaptation. The key challenge is transferring the learned residual policy despite the visual gap between simulation and reality. Rather than requiring pixel-level visual correspondence, VLaRL uses the VLA's internal vision-language latent representation to condition residual control and as the sim-to-real transfer interface, and learns a lightweight mapper that transforms simulation-derived latents toward the real latent distribution. Across four contact-rich manipulation tasks and two VLA backbones, VLaRL improves real-world success in all task-backbone combinations, while controlled ablations demonstrate the importance of both latent conditioning and latent alignment for transferring simulation-trained residual control.

The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.

More in Robotics

• arXiv preprint · 25 September 2026

  MR. POP: Multi-Robot Parallel Optimizing Planner for Almost-Surely Asymptotically Optimal Planning

  Chih H. Huang, Roy Xing, Brian Plancher and 1 other
• arXiv preprint · 26 September 2026

  Active 3D weaves for load-bearing and damage-resilient locomotion

  Guowei Wayne Tu and Evgueni T. Filipov
• arXiv preprint · 25 September 2026

  Can a Robot Read Braille? - Learning to Adapt Contact via Imitation Learning for Tactile Braille Recognition

  Xi Chen, Yunlong Shan, Sihan Chen and 7 others
• arXiv preprint · 26 September 2026

  SAMBAR: Selective Anchoring via Method of Multipliers for Balanced Knowledge Acquisition and Retention in Vision-Language-Action Models

  Aayushi Shrivastava, Xunlan Zhou, Hongrui Zhao and 2 others
• arXiv preprint · 25 September 2026

  Multi-Objective Human-in-the-Loop Bayesian Optimization of a Lower-Limb Exoskeleton

  Neil Janwani, Matthew T. Lerner, Aaron J. Young and 1 other

All new papers in Robotics