単一デモンストレーションからの強化学習における逐次性の活用
Leveraging Sequentiality in Reinforcement Learning from a Single Demonstration
単一のデモンストレーションから逐次的な目標条件付き方策を学習し、ヒューマノイドの歩行や立ち上がりなどの複雑なロボットタスクを高いサンプル効率で解決する手法を提案。
著者: Alexandre Chenu, Olivier Serris, Olivier Sigaud, Nicolas Perrin-Gilbert
分類: cs.RO, cs.AI
原文アブストラクト
Deep Reinforcement Learning has been successfully applied to learn robotic control. However, the corresponding algorithms struggle when applied to problems where the agent is only rewarded after achieving a complex task. In this context, using demonstrations can significantly speed up the learning process, but demonstrations can be costly to acquire. In this paper, we propose to leverage a sequential bias to learn control policies for complex robotic tasks using a single demonstration. To do so, our method learns a goal-conditioned policy to control a system between successive low-dimensional goals. This sequential goal-reaching approach raises a problem of compatibility between successive goals: we need to ensure that the state resulting from reaching a goal is compatible with the achievement of the following goals. To tackle this problem, we present a new algorithm called DCIL-II. We show that DCIL-II can solve with unprecedented sample efficiency some challenging simulated tasks such as humanoid locomotion and stand-up as well as fast running with a simulated Cassie robot. Our method leveraging sequentiality is a step towards the resolution of complex robotic tasks under minimal specification effort, a key feature for the next generation of autonomous robots.
