強化学習で拡張したMPCによる俊敏で堅牢な二足歩行の足運び計画と制御
RL-augmented MPC Framework for Agile and Robust Bipedal Footstep Locomotion Planning and Control
ALIPベースのMPCで大まかな足運びを計画し、学習方策で足位置を微調整することで、全身ダイナミクスを考慮した俊敏で堅牢な二足歩行を実現する枠組みを提案し、ヒューマノイドDRACO 3で検証した。
著者: Seung Hyeon Bang, Carlos Arribalzaga Jové, Luis Sentis
分類: cs.RO
原文アブストラクト
This paper proposes an online bipedal footstep planning strategy that combines model predictive control (MPC) and reinforcement learning (RL) to achieve agile and robust bipedal maneuvers. While MPC-based foot placement controllers have demonstrated their effectiveness in achieving dynamic locomotion, their performance is often limited by the use of simplified models and assumptions. To address this challenge, we develop a novel foot placement controller that leverages a learned policy to bridge the gap between the use of a simplified model and the more complex full-order robot system. Specifically, our approach employs a unique combination of an ALIP-based MPC foot placement controller for sub-optimal footstep planning and the learned policy for refining footstep adjustments, enabling the resulting footstep policy to capture the robot's whole-body dynamics effectively. This integration synergizes the predictive capability of MPC with the flexibility and adaptability of RL. We validate the effectiveness of our framework through a series of experiments using the full-body humanoid robot DRACO 3. The results demonstrate significant improvements in dynamic locomotion performance, including better tracking of a wide range of walking speeds, enabling reliable turning and traversing challenging terrains while preserving the robustness and stability of the walking gaits compared to the baseline ALIP-based MPC approach.
