FIT-MAC: A Closed-Loop Multi-Agent Framework for Robust Vision-Language-Action Models

Abstract

Vision-language-action (VLA) models have achieved strong performance on standard robotic manipulation benchmarks. However, their reliability can degrade substantially when task conditions deviate from the nominal setting. Perturbations in object positions and task instructions may lead to complete task failure. Existing VLA policies generally lack an explicit runtime mechanism for distinguishing these failure modes and selecting an appropriate recovery strategy. To systematically mitigate this issue, we propose FIT-MAC, a failure-aware, closed-loop multi-agent control framework for improving the robustness of VLA policies at inference time. FIT-MAC augments a VLA policy with an inference-time control layer. First, the Planner Agent parses the language instruction into a structured task specification that captures the manipulation goal and spatial constraints, and the Grounder Agent resolves each instruction-level reference to a visual target. Next, the Verifier Agent tracks phase-specific execution progress, distinguishes target-binding errors from local execution failures, and determines whether intervention is warranted. The Correction Agent then routes permitted interventions to target rebinding or bounded grasping or placement recovery, depending on the failure type. When no intervention is required, the pretrained VLA remains the default action generator. Extensive experiments on LIBERO-PRO demonstrate that FIT-MAC improves robustness across multiple VLA backbones, consistently increasing success rates and achieving state-of-the-art performance under diverse perturbations.

The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.

More in Robotics

• arXiv preprint · 20 October 2026

  Real-Time Disturbance-Aware Koopman Model Predictive Control for Quadrotors in Spatially Varying Winds

  Stella Kombo, Masih Haseli, Alejandro A. Stefan-Zavala and 3 others
• arXiv preprint · 21 October 2026

  PHD-Hand: A Printable Humanoid Dexterous Hand with Linkage-Driven 3-DOA Digits

  Neel Adwani, Muhaiminul Islam Akash, Saffia Ouali and 2 others
• arXiv preprint · 20 October 2026

  Towards Autonomous Bricklaying Using Boston Dynamics Spot

  Karl-Johan Sorensen, Nico vom Hofe and Frank H. P. Fitzek
• arXiv preprint · 21 October 2026

  Causal world models under physical parameter shifts

  Chu Lin and Yi Sun
• arXiv preprint · 20 October 2026

  HATCH: Hardware-Attested Task Chains for Verifiable Manipulation

  Shiza Charania

All new papers in Robotics