見極めることを学ぶ：選好と表現学習による異種人間デモンストレーションの模倣
Learning to Discern: Imitating Heterogeneous Human Demonstrations with Preference and Representation Learning
品質やスタイルが異なる人間のデモンストレーションから学ぶため、軌道セグメントの潜在表現と選好学習を用いて品質評価器を訓練するオフライン模倣学習フレームワークL2Dを提案し、シミュレーションと実機で有効性を示した。
著者: Sachit Kuhar, Shuo Cheng, Shivang Chopra, Matthew Bronars, Danfei Xu
分類: cs.RO, cs.AI
原文アブストラクト
Practical Imitation Learning (IL) systems rely on large human demonstration datasets for successful policy learning. However, challenges lie in maintaining the quality of collected data and addressing the suboptimal nature of some demonstrations, which can compromise the overall dataset quality and hence the learning outcome. Furthermore, the intrinsic heterogeneity in human behavior can produce equally successful but disparate demonstrations, further exacerbating the challenge of discerning demonstration quality. To address these challenges, this paper introduces Learning to Discern (L2D), an offline imitation learning framework for learning from demonstrations with diverse quality and style. Given a small batch of demonstrations with sparse quality labels, we learn a latent representation for temporally embedded trajectory segments. Preference learning in this latent space trains a quality evaluator that generalizes to new demonstrators exhibiting different styles. Empirically, we show that L2D can effectively assess and learn from varying demonstrations, thereby leading to improved policy performance across a range of tasks in both simulations and on a physical robot.
