最適輸送による観察からの模倣学習
Imitation Learning from Observation through Optimal Transport
学習者と専門家の状態軌道間のワッサースタイン距離に基づく報酬を、学習モデルや敵対的学習なしで生成する簡素化された観察からの模倣学習手法を提案し、連続制御タスクで専門家レベルの性能を達成した。
著者: Wei-Di Chang, Scott Fujimoto, David Meger, Gregory Dudek
分類: cs.RO, cs.AI, cs.LG
原文アブストラクト
Imitation Learning from Observation (ILfO) is a setting in which a learner tries to imitate the behavior of an expert, using only observational data and without the direct guidance of demonstrated actions. In this paper, we re-examine optimal transport for IL, in which a reward is generated based on the Wasserstein distance between the state trajectories of the learner and expert. We show that existing methods can be simplified to generate a reward function without requiring learned models or adversarial learning. Unlike many other state-of-the-art methods, our approach can be integrated with any RL algorithm and is amenable to ILfO. We demonstrate the effectiveness of this simple approach on a variety of continuous control tasks and find that it surpasses the state of the art in the IlfO setting, achieving expert-level performance across a range of evaluation domains even when observing only a single expert trajectory without actions.
