MILE: モデルベース介入学習
MILE: Model-based Intervention Learning
専門家の介入データと非介入時のフィードバックを活用し、モデルベースで介入を定式化することで、わずかな介入からロボットの模倣学習を効率的に行う手法を提案。
著者: Yigit Korkmaz, Erdem Bıyık
分類: cs.RO, cs.AI, cs.LG
原文アブストラクト
Imitation learning techniques have been shown to be highly effective in real-world control scenarios, such as robotics. However, these approaches not only suffer from compounding error issues but also require human experts to provide complete trajectories. Although there exist interactive methods where an expert oversees the robot and intervenes if needed, these extensions usually only utilize the data collected during intervention periods and ignore the feedback signal hidden in non-intervention timesteps. In this work, we create a model to formulate how the interventions occur in such cases, and show that it is possible to learn a policy with just a handful of expert interventions. Our key insight is that it is possible to get crucial information about the quality of the current state and the optimality of the chosen action from expert feedback, regardless of the presence or the absence of intervention. We evaluate our method on various discrete and continuous simulation environments, a real-world robotic manipulation task, as well as a human subject study. Videos and the code can be found at https://liralab.usc.edu/mile .
