GR2PO: Group Relative Return Policy Optimization for Continuous Robot Control

Abstract

Actor-critic architecture has been widely used in continuous robot control. However, they rely on learning a value network, introducing additional computational overhead during training. Moreover, policy learning may also be affected by the approximation error of value estimation. Critic-free group relative policy optimization methods provide a simpler training approach by removing the need for a critic. However, they fail to learn long-term action outcomes when directly applying immediate rewards to policy optimization in dense-reward environments. To address these problems, we propose Group Relative Return Policy Optimization (GR2PO), a critic-free reinforcement learning framework for continuous robot control. GR2PO estimates the discounted returns from the parallelly collected trajectories, performs group normalization at each rollout time index, and uses relative advantages and clipped targets to update the policy. To evaluate the effectiveness of the proposed framework, we instantiate it on robot control simulation environments and deploy the model to a real-world edge device. The results show that GR2PO significantly outperforms critic-free baselines that use immediate rewards and performs competitively against state-of-the-art actor-critic methods. Furthermore, GR2PO demonstrates competitive training efficiency. Inference tests on NVIDIA Jetson TX2 demonstrate the feasibility of deploying the learned policies on edge platforms. Further ablation experiments analyze the effects of parallel group size, return estimation methods, and target clipping ratio on learning performance. To support follow-up research, we will make the complete code publicly available after the paper is accepted, including the framework implementation, experimental configuration, and training and evaluation scripts.

The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.

More in Robotics

• arXiv preprint · 17 September 2026

  CoreSense: Traceable Failure Recall and Conflict-Aware Belief Gating for Auditable Robot Decisions

  Zoe Li
• arXiv preprint · 18 September 2026

  MA-LIPP: Cooperative Multi-Agent Load-Aware Informative Path Planning for Heterogeneous Robot Teams

  Hojune Kim, Guangyao Shi and Gaurav S. Sukhatme
• arXiv preprint · 17 September 2026

  AMB3R-SLAM: Kilometer-scale SLAM with Hierarchical Backend

  Hengyi Wang and Lourdes Agapito
• arXiv preprint · 18 September 2026

  OpenRoIS: A Community-Driven Open-Source Middleware Implementing the Robotic Interaction Service (RoIS) Framework for Physical Robots and Virtual Agents

  Sebastian Carrera Villalobos, Christopher Nolan Arellano, Arne Hitzmann and 5 others
• arXiv preprint · 17 September 2026

  AURORA: A Natural Language-Driven Agentic Framework for Understanding, Reasoning, and Orchestrating Reliable Air-Ground Co-Simulation

  Keshu Wu, Hao Zhang, Rui Gan and 4 others

All new papers in Robotics