LoopQ: Fair, Closed-Loop Evaluation of Quantized Vision-Language-Action Models

Abstract

Post-training quantization (PTQ) for vision-language-action (VLA) models promises smaller and faster VLA deployments. However, when analyzing existing VLA quantization works, we realised that it is hard to reason about the performance of the suggested quantization method compared to existing, similar, or in many cases, simpler PTQ methods. We present LoopQ, a closed-loop benchmark that compares multiple quantization policies on the same checkpoint, layers and engine, varying only the method and the kernels. We evaluate RTN, AWQ, GPTQ, ActQuant and QuantVLA on four VLAs with more than 180,000 episodes of LIBERO and LIBERO-plus, compare against QVLA's reported results on the same layers, and benchmark whether the four public VLA-quantization releases can be reproduced. We benchmark the methods on an NVIDIA H100, an L4, and a Jetson AGX Thor. Our results show that while quantization reliably saves memory, 2-3x for real low-bit models, VLA-specific PTQ methods underperform compared to simpler generic baselines. At 4 bits, RTN, AWQ and GPTQ match or exceed ActQuant and QuantVLA, plain RTN matches the calibrated methods, and plain AWQ matches the success QVLA reports for itself. Standard LIBERO hides the losses that do exist: configurations with no significant LIBERO loss lose 1.5-4.3 points under paired LIBERO-plus perturbations. Speed is a weaker story. At batch 1 on the H100, no quantized pi0.5 or OpenVLA-OFT configuration is faster than its 16-bit model (1.06-1.58x slower on library kernels); only the L4 with int8 kernels gains (4-14%), the Thor is slower on every 4-bit configuration, and the engine alone changes latency by 2.1x. Profiling explains why: pi0.5's quantized backbone is compute-bound, while its memory-bound action expert, 63% of GPU time and 13% of the parameters, stays in 16 bits.

The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.

More in Robotics

• arXiv preprint · 3 November 2026

  VARP: A Visual Adaptive Rearrangement Planner for Non-Monotone Object Rearrangement Planning

  Hanwen Ren, Meng Zheng, Zhongpai Gao and 4 others
• arXiv preprint · 4 November 2026

  Closed-Loop Adaptive Sampling on a Generative Ocean Model

  Shayesteh Hafezi and Alan Papalia
• arXiv preprint · 3 November 2026

  Online Hamilton-Jacobi Reachability Onboard the Robot Control Loop: A Real-Time GPU Solver for Safety-Critical Autonomy

  Kevin Weekly and Alexandre M. Bayen
• arXiv preprint · 4 November 2026

  ZipDrift: Generating Multimodal Short-Horizon MPC Costs with One-Step Drifting Models

  Ke Zhang
• arXiv preprint · 3 November 2026

  Say the Same, Act Differently: Diagnosing and Steering Text-Action Divergence in Reasoning VLAs

  Zihao Feng, Qingzhao Zhang, Chunyu Xia and 3 others

All new papers in Robotics