暗黙的・双方向カリキュラムによるデモンストレーション不要の自律強化学習
Demonstration-free Autonomous Reinforcement Learning via Implicit and Bidirectional Curriculum
デモンストレーションやリセットなしで非エピソード的に学習する自律強化学習手法を提案し、学習進捗に応じて起動する補助エージェントと最適輸送に基づく双方向ゴールカリキュラムで従来手法を上回った。
著者: Jigang Kim, Daesol Cho, H. Jin Kim
分類: cs.LG, cs.AI, cs.RO
原文アブストラクト
While reinforcement learning (RL) has achieved great success in acquiring complex skills solely from environmental interactions, it assumes that resets to the initial state are readily available at the end of each episode. Such an assumption hinders the autonomous learning of embodied agents due to the time-consuming and cumbersome workarounds for resetting in the physical world. Hence, there has been a growing interest in autonomous RL (ARL) methods that are capable of learning from non-episodic interactions. However, existing works on ARL are limited by their reliance on prior data and are unable to learn in environments where task-relevant interactions are sparse. In contrast, we propose a demonstration-free ARL algorithm via Implicit and Bi-directional Curriculum (IBC). With an auxiliary agent that is conditionally activated upon learning progress and a bidirectional goal curriculum based on optimal transport, our method outperforms previous methods, even the ones that leverage demonstrations.
