スケーラブルな動作模倣による四足ロボットの多様な行動を実現する単一ポリシーの学習
Learning a Single Policy for Diverse Behaviors on a Quadrupedal Robot using Scalable Motion Imitation
多数の参照動作を模倣する深層強化学習により、歩行・旋回・跳躍・座る・伏せるなど多様な行動をこなす単一ポリシーを四足ロボット向けに学習させ、適応的動作サンプリングで難易度の高いスキルへの集中と忘却防止を両立した。
著者: Arnaud Klipfel, Nitish Sontakke, Ren Liu, Sehoon Ha
分類: cs.RO
原文アブストラクト
Learning various motor skills for quadrupedal robots is a challenging problem that requires careful design of task-specific mathematical models or reward descriptions. In this work, we propose to learn a single capable policy using deep reinforcement learning by imitating a large number of reference motions, including walking, turning, pacing, jumping, sitting, and lying. On top of the existing motion imitation framework, we first carefully design the observation space, the action space, and the reward function to improve the scalability of the learning as well as the robustness of the final policy. In addition, we adopt a novel adaptive motion sampling (AMS) method, which maintains a balance between successful and unsuccessful behaviors. This technique allows the learning algorithm to focus on challenging motor skills and avoid catastrophic forgetting. We demonstrate that the learned policy can exhibit diverse behaviors in simulation by successfully tracking both the training dataset and out-of-distribution trajectories. We also validate the importance of the proposed learning formulation and the adaptive motion sampling scheme by conducting experiments.
