DexTaG: Tactile-as-Guidance in Reinforcement Learning for Dexterous Manipulation

Abstract

Glove-based motion capture is emerging as a scalable approach to collecting dexterous-hand demonstration data. However, due to the kinematic gap between the human and robot hand, the recorded human motions cannot be executed directly on the robot, especially for contact-rich tool-use tasks involving in-hand reorientation. Prior work bridges this gap in simulation through reinforcement learning (RL) or trajectory optimization, but the human contact pattern is hard to preserve under such formulations, often producing unnatural manipulation and unstable functional grasps. These methods also train a separate policy or solve a separate optimization for each reference trajectory, which is inefficient. To solve these problems, we propose DexTaG, a tactile-guided RL framework for dexterous manipulation. During training, tactile signals captured by the glove guide policy search toward the measured human contact pattern, reducing reliance on precise reference geometry for contact supervision. To improve efficiency, we train a single generalizable retargeter jointly on all training trajectories of the same object. The retargeter is further distilled into a tactile-free student controller conditioned on the target object trajectory for real-world deployment. On marker-pen and hammer manipulation tasks, DexTaG learns natural, contact-rich behaviors that baselines with distance-based contact heuristics fail to learn, generalizes to held-out trajectories of the same object and task, and outperforms single-trajectory baselines on OakInk2.

The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.

More in Robotics

• arXiv preprint · 27 September 2026

  Is Online Interaction Necessary for Recovery? A Minimalist Approach to Robust Planning via Perturbation

  Bumgeun Park and Donghwan Lee
• arXiv preprint · 28 September 2026

  Quantile Head for Vision-Language-Action Models

  Xuan Wang, Yinan Wu, Haoran Duan and 1 other
• arXiv preprint · 27 September 2026

  SwingRL: Adaptive Observation Reinforcement Learning with World-Model Prediction for Cable-Suspended Hoisting Control

  Guangming Wang, Xiaoyu Zhang, Yucheng Xin and 8 others
• arXiv preprint · 28 September 2026

  AD-E2E-JEPA: A Joint-Embedding Predictive Architecture For End-to-End Autonomous Driving

  Haoran Zhu, Wancong Zhang, Yann LeCun and 1 other
• arXiv preprint · 27 September 2026

  REALM: A Coarse-to-Fine Generative Framework for Embodied Reactive Listening

  Peizhen Li, Longbing Cao and Yang Zhang

All new papers in Robotics