Behavioral Steering of Vision-Language-Action Models in the Low-Parameter Regime

Abstract

Reinforcement learning (RL) is increasingly used to adapt pretrained vision-language-action models (VLAs). We investigate the problem of behavioral steering: adapting a VLA's execution style to preferences such as carrying objects higher. To investigate how little trainable capacity such steering requires and how much pretrained capability can be retained, we freeze an OpenVLA-OFT 7B policy and optimize TinyLoRA adapters with 8 to 384 trainable parameters using either policy-gradient RL (GRPO) or evolutionary search (CMA-ES). With just 256 trainable parameters, we raise carry height by 120 mm on trained tasks at 92% success and by 74 mm on a held-out task at 80% success. By rescaling the learned update, we can tune the trade-off between style gains and capability retention without retraining. Increasing adapter capacity generally improves this trade-off, retaining more pretrained capability at comparable style gains. The choice of optimizer also depends on capacity, with CMA-ES being competitive at low parameter counts, while GRPO offers better trade-offs as capacity grows. These findings show that extremely low-dimensional updates can steer pretrained VLA behavior, with adapter capacity, optimizer choice, and update scaling offering practical control over adaptation and retention.

The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.

More in Robotics

• arXiv preprint · 31 October 2026

  A Straight Line Is Not a Camera Pose: Auditing Short-Clip Pose Benchmarks

  Xinyi Shi
• arXiv preprint · 1 November 2026

  Same Path, Less Time: Training-Free Fast Execution of Vision-Language-Action Models under Inference Latency

  Fuyi Chen
• arXiv preprint · 31 October 2026

  Knowing Where to Look: Interactive Object Search via Spatial-Semantic Fusion

  Aditya Agarwal, Lucy Cai, Kelly Lu and 4 others
• arXiv preprint · 1 November 2026

  Serving Agentic Robot Fleets: Workload Characterization, KV-Cache Thrashing, and Admission Control

  Sida He
• arXiv preprint · 31 October 2026

  MESSPa: A Multi-modal Embodiment Stimulation Shoulder Pad for Preserving the Sense of Agency in a Detachable Robotic Arm

  Vitvasin Vimolmongkolporn, Guillaume Gourmelen, Yukiko Iwasaki and 1 other

All new papers in Robotics