AWLS: Advantage-Weighted Latent Steering for One-Step Generative Robotic Manipulation Policies

Abstract

Human-collected real-world robotic manipulation demonstrations often contain mixed-quality trajectories, including hesitant, indirect, or inefficient action segments. While diffusion- and flow-based policies can model such multimodal data, standard behavior cloning (BC) assigns equal likelihood to all demonstrated actions and may therefore reproduce suboptimal modes. Offline RL value estimates offer a natural signal for biasing policies toward higher-value behavioral modes, but directly fine-tuning the generative policy or using critic gradients for policy steering can amplify critic errors and induce prior drift. We propose Advantage-Weighted Latent Steering (AWLS), an offline post-training method for improving frozen one-step generative manipulation policies without updating their decoders. AWLS learns a residual noise shaper, an observation-conditioned residual correction to the policy's latent noise, and trains it with an advantage-weighted action reconstruction objective. The advantages are detached, scale-normalized, and used only as supervised weights on demonstrated actions, avoiding critic-gradient policy updates while preserving fast single-step inference. We further introduce a rollout-free decoder-sensitivity criterion for selecting pretrained checkpoints that remain responsive to latent-noise perturbations. Experiments on simulated and real-world robotic manipulation tasks with clean and mixed-quality demonstrations show that AWLS improves over the frozen base policy while limiting drift from the pretrained behavior prior. Project page: https://awls-corl2026.github.io/

The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.

More in Robotics

• arXiv preprint · 20 October 2026

  Real-Time Disturbance-Aware Koopman Model Predictive Control for Quadrotors in Spatially Varying Winds

  Stella Kombo, Masih Haseli, Alejandro A. Stefan-Zavala and 3 others
• arXiv preprint · 21 October 2026

  PHD-Hand: A Printable Humanoid Dexterous Hand with Linkage-Driven 3-DOA Digits

  Neel Adwani, Muhaiminul Islam Akash, Saffia Ouali and 2 others
• arXiv preprint · 20 October 2026

  Towards Autonomous Bricklaying Using Boston Dynamics Spot

  Karl-Johan Sorensen, Nico vom Hofe and Frank H. P. Fitzek
• arXiv preprint · 21 October 2026

  Causal world models under physical parameter shifts

  Chu Lin and Yi Sun
• arXiv preprint · 20 October 2026

  HATCH: Hardware-Attested Task Chains for Verifiable Manipulation

  Shiza Charania

All new papers in Robotics