Permutation Robustness Is Not Enough: Action Collapse in Multi-Agent Transformer Policies

Abstract

Transformer policies are attractive for multi-agent robot learning because self-attention can model interactions among agents. However, multi-agent teams are unordered, while transformers typically process agents as ordered token sequences. We study how this mismatch affects cooperative navigation policies under agent-order permutations. Our results show that low permutation error alone can be misleading: policies may appear robust simply because all agents choose the same action. We therefore evaluate policies using both permutation-consistency metrics and action-collapse diagnostics, including action diversity, same-action fraction, and maximum action frequency. A PPO-ID baseline yields non-collapsed behavior but remains order-sensitive, while strong equivariance regularization can still induce homogeneous behavior. A weak equivariance penalty improves the robustness while preserving more diverse actions for teams with \(N=3\) agents, whereas teams with \(N=4\) agents require substantially smaller regularization weights. These findings suggest that multi-agent transformer policies should be evaluated not only by return and permutation robustness, but also by whether they maintain non-collapsed, differentiated multi-agent behavior.

The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.

More in Robotics

• arXiv preprint · 2 October 2026

  WebUIProof: Benchmarking WebUI Code Generators with UI-Agent Execution Harness

  Yun-Yun Tsai, Yuning Mao, Shiqi Wang and 2 others
• arXiv preprint · 3 October 2026

  Playing social deduction games with reinforcement fine-tuned large language models

  Lingzhe Zhang, Yunpeng Zhai, Tong Jia and 7 others
• arXiv preprint · 2 October 2026

  Performance of Zero-determinant Strategies in Repeated Games without Discounting

  Masahiko Ueda
• arXiv preprint · 3 October 2026

  Evidence and Intervention: A Coupled Active-Inference Extension of Rational Speech Act Models

  Yonghyeon Gwon, Elliot Murphy and Chun Kee Chung
• arXiv preprint · 2 October 2026

  Coherence-Driven Belief Formation and Population Dynamics of Contagion in LLM Agents

  Tathagata Banerjee and Nima Moghaddas

All new papers in Robotics