不完全なデモンストレーションからの模倣学習：動特性が異なるエージェントへの対応
Learning from Imperfect Demonstrations from Agents with Varying Dynamics
最適でない、あるいは動特性の異なるエージェントが生成したデモンストレーションから模倣学習するため、実現可能性スコアと最適性スコアからなる指標を開発し、有用なデモを選択的に活用する手法を提案した。
著者: Zhangjie Cao, Dorsa Sadigh
分類: cs.LG, cs.RO
原文アブストラクト
Imitation learning enables robots to learn from demonstrations. Previous imitation learning algorithms usually assume access to optimal expert demonstrations. However, in many real-world applications, this assumption is limiting. Most collected demonstrations are not optimal or are produced by an agent with slightly different dynamics. We therefore address the problem of imitation learning when the demonstrations can be sub-optimal or be drawn from agents with varying dynamics. We develop a metric composed of a feasibility score and an optimality score to measure how useful a demonstration is for imitation learning. The proposed score enables learning from more informative demonstrations, and disregarding the less relevant demonstrations. Our experiments on four environments in simulation and on a real robot show improved learned policies with higher expected return.
