4.5分のデータで四足歩行を実現するモデルベース強化学習
Data Efficient Reinforcement Learning for Legged Robots
四足ロボットの歩行をわずか4.5分の実機データで学習するモデルベース手法を提案し、複数ステップ予測損失と遅延対応MPC、脚軌道の事前知識でリアルタイムかつ安全な歩行を実現した。
著者: Yuxiang Yang, Ken Caluwaerts, Atil Iscen, Tingnan Zhang, Jie Tan, Vikas Sindhwani
分類: cs.LG, cs.AI, cs.RO
原文アブストラクト
We present a model-based framework for robot locomotion that achieves walking based on only 4.5 minutes (45,000 control steps) of data collected on a quadruped robot. To accurately model the robot's dynamics over a long horizon, we introduce a loss function that tracks the model's prediction over multiple timesteps. We adapt model predictive control to account for planning latency, which allows the learned model to be used for real time control. Additionally, to ensure safe exploration during model learning, we embed prior knowledge of leg trajectories into the action space. The resulting system achieves fast and robust locomotion. Unlike model-free methods, which optimize for a particular task, our planner can use the same learned dynamics for various tasks, simply by changing the reward function. To the best of our knowledge, our approach is more than an order of magnitude more sample efficient than current model-free methods.
