LEAP: Making Privileged Geometry Supervision Effective for Visuomotor Learning

Abstract

Privileged 3D supervision uses additional geometric information during training to guide RGB-based visuomotor policy learning, without requiring geometric inputs at deployment. However, low reconstruction error does not ensure that visual representations capture geometry useful for control. We identify three limitations that weaken this supervision: proprioceptive shortcuts, dominant-view reliance, and reconstruction objectives dominated by task-irrelevant geometry. To address these limitations, we propose Latent Encoding with Aligned Privileged Geometry (LEAP). Our framework uses an auxiliary decoder to reconstruct point clouds within the manipulation workspace from visual features alone, while retaining proprioception for action prediction. Alongside full reconstruction, we introduce wrist-view dropout and partial reconstruction targets matched to the retained wrist, encouraging the encoder to capture complementary local geometry. The auxiliary decoder is removed at inference, leaving only RGB observations and robot state as policy inputs. Extensive experiments on RoboTwin, ManiSkill, and real-world tasks demonstrate consistent and substantial improvements over Diffusion Policy and ACT, with only a small increase in parameter count and no reduction in inference speed.

The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.

More in Robotics

• arXiv preprint · 4 October 2026

  Tackling Sim-to-Real Mismatch Through Sampling-Based Disturbance Observers: From Analytical Models to Learned World Models

  Tianqi Zhu, Jun Yang, Jianliang Mao and 2 others
• arXiv preprint · 5 October 2026

  FOCUS: Fine-Grained Open-Vocabulary Change Detection for Uncertainty-Aware Semi-Static Scenes

  Can Xu, Mingfeng Yuan, Mahan Mohammadi and 1 other
• arXiv preprint · 4 October 2026

  GJK-CBF: Control Barrier Functions for Convex Rigid Body Collision Avoidance on SE(3)

  Yi-Hsuan Chen, Shuo Liu, Wei Xiao and 2 others
• arXiv preprint · 5 October 2026

  Spacecraft Rendezvous Trajectory Generation with Modular Constraints via Diffusion Model Composition

  Mariko A. Storey-Matsutani and Richard Linares
• arXiv preprint · 4 October 2026

  $R^2$-WAM: Repair-and-Reject Post-Training for World Action Models

  Ruiyan Xu, Haisheng Su, Sixu Lin and 4 others

All new papers in Robotics