単一動画デモンストレーションからの模倣学習
Towards Learning to Imitate from a Single Video Demonstration
単一の動画デモのみから報酬関数を対照学習で獲得し、強化学習エージェントが状態・行動情報なしに模倣行動を学習する手法を提案した論文。
著者: Glen Berseth, Florian Golemo, Christopher Pal
分類: cs.LG, cs.RO, stat.ML
原文アブストラクト
Agents that can learn to imitate given video observation -- \emph{without direct access to state or action information} are more applicable to learning in the natural world. However, formulating a reinforcement learning (RL) agent that facilitates this goal remains a significant challenge. We approach this challenge using contrastive training to learn a reward function comparing an agent's behaviour with a single demonstration. We use a Siamese recurrent neural network architecture to learn rewards in space and time between motion clips while training an RL policy to minimize this distance. Through experimentation, we also find that the inclusion of multi-task data and additional image encoding losses improve the temporal consistency of the learned rewards and, as a result, significantly improves policy learning. We demonstrate our approach on simulated humanoid, dog, and raptor agents in 2D and a quadruped and a humanoid in 3D. We show that our method outperforms current state-of-the-art techniques in these environments and can learn to imitate from a single video demonstration.
