Value-Preserving Action Quantization

Abstract

Search-based policy learning, such as Monte Carlo tree search (MCTS), offers the flexibility to trade runtime computation for data efficiency and adaptation to novel environments. For robot learning, however, applying search requires discretizing the continuous action space and associated dynamics, typically through action quantization. While reconstruction-based quantization is sufficient for representing the continuous action space alone, policy learning additionally requires the quantized actions to preserve their effects on future states and values. We show that such requirement is not necessarily captured by action reconstruction error. Instead, we prove that the error in the value function resulting from action quantization is tightly bounded by the reconstruction error of state reachability under the quantized actions. Based on these results, we propose value-preserving action quantization (VAQ), which quantizes actions by preserving state reachability. We further combine VAQ with MCTS (VAQ-MCTS) as a value-preserving, search-based policy learning framework for behavioral cloning. Using a wide range of object pushing tasks as a controlled benchmark, we evaluate VAQ-MCTS against state-of-the-art behavioral cloning methods. VAQ-MCTS matches or exceeds their performance under standard settings, while substantially improving data efficiency and robustness to heterogeneous demonstrations, while enabling zero-shot adaptation to novel environments. Hardware experiments further demonstrate the effectiveness of VAQ-MCTS, completing all 30 real-world object pushing trials with only 10 minutes of demonstration data.

The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.

More in Robotics

• arXiv preprint · 12 October 2026

  PathLift: Depth-Conditioned Metric Lifting for Visual-Navigation Priors

  Gwanjun Shin, Hyeongseok Suh, Hongmo Jung and 1 other
• arXiv preprint · 13 October 2026

  An Optimal Algorithm for Changing from Longitudinal to Latitudinal Formation of Autonomous Aircraft Squadrons

  Paulo André Sperandio Giacomin and Elder Moreira Hemerly
• arXiv preprint · 12 October 2026

  Constrained Differentiable Policy Learning for Safe Quadrotor Navigation

  Yan Rui Tan, Praveen Krishnapur, Zeyu Feng and 3 others
• arXiv preprint · 13 October 2026

  YUBI-Real2Sim: 3DGS Scene Verification for Reliable Policy Evaluation

  Hiroki Nishizawa, Takehiko Ohkawa, Nicola Scianca and 3 others
• arXiv preprint · 12 October 2026

  Directional Pullback Regularization for Long-Horizon Rollout of Latent World Models in Autonomous Driving

  Shatadal Mishra, Ahmadreza Moradipari and Nejib Ammar

All new papers in Robotics