arXiv preprint ·
Improving Policy Learning from Delayed Interventions
- Published
- Source
- arXiv preprint, posted before peer review
- Original
- Read it at arXiv
- Topics
Abstract
In this work, we explore the idea of expert intervention delays in human-supervised robotic systems for imitation learning system. Previous works in imitation learning have primarily examined the impact of suboptimal experts, where the expert policy makes mistakes, but assumed expert interventions occur immediately after errors are made. Although delayed expert intervention can be viewed as a form of noise that might improve policy robustness, our results show that learning from delayed expert interventions is overall less effective than learning from non-delayed expert interventions; delayed expert interventions are less effective at demonstrating error avoidance behaviors. To address this, we propose Learning from Delayed Intervention (LfDI), which learns a cost function based on where and when the expert intervenes, combined with limited information about the extent of intervention delays provided by the expert. This cost function is then used to infer additional supervision for states without expert data, guiding the policy to avoid states that would likely trigger interventions. We evaluate our method, Learning from Delayed Intervention (LfDI), on simulated environments and show that it outperforms other methods that do not account for intervention delays.
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Robotics
PathLift: Depth-Conditioned Metric Lifting for Visual-Navigation Priors
An Optimal Algorithm for Changing from Longitudinal to Latitudinal Formation of Autonomous Aircraft Squadrons
Constrained Differentiable Policy Learning for Safe Quadrotor Navigation
YUBI-Real2Sim: 3DGS Scene Verification for Reliable Policy Evaluation
Directional Pullback Regularization for Long-Horizon Rollout of Latent World Models in Autonomous Driving
Get the few papers worth your time in Robotics every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.