不完全なデモンストレーションからの敵対的信頼度転移による学習
Learning from Imperfect Demonstrations via Adversarial Confidence Transfer
信頼度付きのデモがある環境から、ラベルなしデモしかない環境へ敵対的分布マッチングで信頼度予測器を転移し、有益なデモを重み付けして方策を学習する手法を提案した。
著者: Zhangjie Cao, Zihan Wang, Dorsa Sadigh
分類: cs.RO, cs.LG
原文アブストラクト
Existing learning from demonstration algorithms usually assume access to expert demonstrations. However, this assumption is limiting in many real-world applications since the collected demonstrations may be suboptimal or even consist of failure cases. We therefore study the problem of learning from imperfect demonstrations by learning a confidence predictor. Specifically, we rely on demonstrations along with their confidence values from a different correspondent environment (source environment) to learn a confidence predictor for the environment we aim to learn a policy in (target environment -- where we only have unlabeled demonstrations.) We learn a common latent space through adversarial distribution matching of multi-length partial trajectories to enable the transfer of confidence across source and target environments. The learned confidence reweights the demonstrations to enable learning more from informative demonstrations and discarding the irrelevant ones. Our experiments in three simulated environments and a real robot reaching task demonstrate that our approach learns a policy with the highest expected return.
