モデルベース監督による強化学習で効率的に頑健なトルクベース二足歩行を学習
Efficiently Learning Robust Torque-based Locomotion Through Reinforcement with Model-Based Supervision
モデルベースの二足歩行制御に残差強化学習を組み合わせ、訓練時に真のダイナミクスを使えるオラクル方策で残差方策を監督することで、報酬設計を複雑にせず実世界の不確かさに頑健な歩行を効率的に学習する手法を提案。
詳しい要約
1. どんなもの?
2. 先行研究と比べてどこがすごい?
3. 技術・手法の肝は?
4. どうやって有効だと検証した?
5. 議論はある?
6. 次に読むべき論文は?
※ AIが要旨から生成した要約です。正確性は原文をご確認ください。
著者: Yashuai Yan, Tobias Egle, Christian Ott, Dongheui Lee
分類: cs.RO
原文アブストラクト
We propose a control framework that integrates model-based bipedal locomotion with residual reinforcement learning (RL) to achieve robust and adaptive walking in the presence of real-world uncertainties. Our approach leverages a model-based controller, comprising a Divergent Component of Motion (DCM) trajectory planner and a whole-body controller, as a reliable base policy. To address the uncertainties of inaccurate dynamics modeling and sensor noise, we introduce a residual policy trained through RL with domain randomization. Crucially, we employ a model-based oracle policy, which has privileged access to ground-truth dynamics during training, to supervise the residual policy via a novel supervised loss. This supervision enables the policy to efficiently learn corrective behaviors that compensate for unmodeled effects without extensive reward shaping. Our method demonstrates improved robustness and generalization across a range of randomized conditions, offering a scalable solution for sim-to-real transfer in bipedal locomotion.
