正例・未ラベル学習による行動クローニングの改善
Improving Behavioural Cloning with Positive Unlabeled Learning
品質が混在するロボットデータセットから少数の高品質な正例を用いて専門家軌道を反復的に特定し、そのフィルタ済みデータに行動クローニングを適用することで、オフライン強化学習や模倣学習のベースラインを上回る性能を実現した。
著者: Qiang Wang, Robert McCarthy, David Cordova Bulens, Kevin McGuinness, Noel E. O'Connor, Nico Gürtler, Felix Widmaier, Francisco Roldan Sanchez, Stephen J. Redmond
分類: cs.LG, cs.RO
原文アブストラクト
Learning control policies offline from pre-recorded datasets is a promising avenue for solving challenging real-world problems. However, available datasets are typically of mixed quality, with a limited number of the trajectories that we would consider as positive examples; i.e., high-quality demonstrations. Therefore, we propose a novel iterative learning algorithm for identifying expert trajectories in unlabeled mixed-quality robotics datasets given a minimal set of positive examples, surpassing existing algorithms in terms of accuracy. We show that applying behavioral cloning to the resulting filtered dataset outperforms several competitive offline reinforcement learning and imitation learning baselines. We perform experiments on a range of simulated locomotion tasks and on two challenging manipulation tasks on a real robotic system; in these experiments, our method showcases state-of-the-art performance. Our website: \url{https://sites.google.com/view/offline-policy-learning-pubc}.
