Training in Deployment: Human-in-the-loop Online Post-Training for Robot Manipulation

Abstract

Large pretrained robot foundation models provide strong priors for manipulation, yet adapting them to target tasks with offline teleoperated demonstrations often fails to cover the contact-rich, recovery, and precision-critical states encountered during deployment. Existing methods mitigate this issue by augmenting offline demonstrations with human-in-the-loop data or online reinforcement learning. Despite this, offline post-training adapts slowly because it cannot immediately absorb deployment experience, while online RL typically relies on value functions or auxiliary actor-critic models, increasing both training complexity and instability. We present TIDE (Training In DEployment), a human-in-the-loop online post-training system. TIDE introduces a provenance-conditioned supervised learning framework that updates the policy online from human interventions and policy-generated rollouts collected during deployment, without a learned critic, reward model, or action scorer. Across four real-world bimanual manipulation tasks, TIDE improves milestone-normalized task progress and binary full-task completion over the warm-start policy and matched additional offline demonstrations, and performs comparably to the fixed-policy collection baseline while making updated checkpoints available during deployment. Each task-specific online post-training session, from the start of deployment rollouts until the final checkpoint is available, completes in less than one hour while learning and checkpoint handling run asynchronously.

The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.

More in Robotics

• arXiv preprint · 21 October 2026

  PHD-Hand: A Printable Humanoid Dexterous Hand with Linkage-Driven 3-DOA Digits

  Neel Adwani, Muhaiminul Islam Akash, Saffia Ouali and 2 others
• arXiv preprint · 22 October 2026

  Know When to Escalate: Critic-Guided Adaptive Execution for Cloud-Local Embodied Agents

  Wei Zhang, Lirong Xiang and Charles Fleming
• arXiv preprint · 21 October 2026

  Causal world models under physical parameter shifts

  Chu Lin and Yi Sun
• arXiv preprint · 22 October 2026

  OASIS: Online Anchor-centric Scene Inference System for Open-Vocabulary Retrieval and Navigation in Dynamic Environments

  Kaiyao Liao, Qingfeng Li, Chen Chen and 3 others
• arXiv preprint · 21 October 2026

  Learning to Absorb Corrections: Critic-Weighted Residual Flow for Vision-Based Insertion

  Yiqing Yuan, Zhi Li, Jinrong Yang and 5 others

All new papers in Robotics