野生動物動画からの強化学習
Reinforcement Learning from Wild Animal Videos
インターネット上の野生動物動画を教師信号として活用し、四足ロボットが歩行・ジャンプ・静止などの多様な運動スキルを獲得する強化学習手法を提案した。
著者: Elliot Chane-Sane, Constant Roux, Olivier Stasse, Nicolas Mansard
分類: cs.RO, cs.CV, cs.LG
原文アブストラクト
We propose to learn legged robot locomotion skills by watching thousands of wild animal videos from the internet, such as those featured in nature documentaries. Indeed, such videos offer a rich and diverse collection of plausible motion examples, which could inform how robots should move. To achieve this, we introduce Reinforcement Learning from Wild Animal Videos (RLWAV), a method to ground these motions into physical robots. We first train a video classifier on a large-scale animal video dataset to recognize actions from RGB clips of animals in their natural habitats. We then train a multi-skill policy to control a robot in a physics simulator, using the classification score of a third-person camera capturing videos of the robot's movements as a reward for reinforcement learning. Finally, we directly transfer the learned policy to a real quadruped Solo. Remarkably, despite the extreme gap in both domain and embodiment between animals in the wild and robots, our approach enables the policy to learn diverse skills such as walking, jumping, and keeping still, without relying on reference trajectories nor skill-specific rewards.
