Scalable Multi-Human Cooperative Manipulation via Commander-Executor Coordination

Abstract

Multi-human cooperative object manipulation is a highly challenging topic in humanoid control. While existing methods can handle simple collaborative carrying tasks, they struggle with role-dependent, long-horizon, and sequential cooperative manipulation. To address this issue, we propose a hierarchical commander-executor framework that enables multiple agents to accomplish complex collaborative tasks in distinct ways. Specifically, a commander combines instructions and scene information for joint semantic and geometric planning. It then decomposes the task into role-specific subtasks and generates shared object-centric coordination plans, specifying execution timing, target positions, and manipulation actions for each agent. Conditioned on the commander's high-level strategy and each agent's local observations, the executor learns a low-level physics-based policy through a solo-to-cooperative curriculum to realize coordinated human-object interactions. The planned object trajectory provides a shared coordination target, while root positions and grasping instructions specify role-dependent execution cues. We evaluate our method on a variety of object manipulation tasks, demonstrating its ability to execute long-horizon tasks that combine different types of collaboration and to scale from two-agent to multi-agent cooperation. Extensive experiments further show that our approach achieves higher success rates compared to state-of-the-art methods. Code will be made publicly available at https://github.com/TJU-3DV/Co-Manipulation.

The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.

More in Robotics

• arXiv preprint · 10 October 2026

  Red-Teaming of Vision-Language-Action Models via Quality Diversity Environment Generation

  Shihan Zhao, Henry Chen, Ya-Chuan Hsu and 7 others
• arXiv preprint · 11 October 2026

  Reinforcement Learning Based Heterogeneous Task Allocation Under Spatio-Temporal Environmental Uncertainty for Multi-Robot Teams

  Xi Lin, Yaxuan Li, Paul Szenher and 1 other
• arXiv preprint · 10 October 2026

  Shared-Control Teleoperation of Omnidirectional Aerial Robots for Continuous Valve Rotation

  Jinjie Li, Yicheng Chen, Haokun Liu and 1 other
• arXiv preprint · 11 October 2026

  LeHami: Symplectic Latent Dynamics for World Models

  Romina Yalovetzky, Randall Balestriero and Yann LeCun
• arXiv preprint · 10 October 2026

  From Field to Planning: Learning Temporal Polar Gradient Fields for Vision-based AAV Trajectory Planning in Dynamic Environments

  Haowen Gao, Zhan Chen, Yinuo Xu and 3 others

All new papers in Robotics