動的バランスモデルで支援するデータ効率と安全性を両立したヒューマノイド歩行学習
Data-Efficient and Safe Learning for Humanoid Locomotion Aided by a Dynamic Balancing Model
歩行パターン生成器とニューラルネット、安全制御器を組み合わせ、動的バランスモデルを活用して少ないデータで安全にヒューマノイドの歩行を強化学習する手法を提案した。
著者: Junhyeok Ahn, Jaemin Lee, Luis Sentis
分類: cs.RO
原文アブストラクト
In this letter, we formulate a novel Markov Decision Process (MDP) for safe and data-efficient learning for humanoid locomotion aided by a dynamic balancing model. In our previous studies of biped locomotion, we relied on a low-dimensional robot model, commonly used in high-level Walking Pattern Generators (WPGs). However, a low-level feedback controller cannot precisely track desired footstep locations due to the discrepancies between the full order model and the simplified model. In this study, we propose mitigating this problem by complementing a WPG with reinforcement learning. More specifically, we propose a structured footstep control method consisting of a WPG, a neural network, and a safety controller. The WPG provides an analytical method that promotes efficient learning while the neural network maximizes long-term rewards, and the safety controller encourages safe exploration based on step capturability and the use of control-barrier functions. Our contributions include the following (1) a structured learning control method for locomotion, (2) a data-efficient and safe learning process to improve walking using a physics-based model, and (3) the scalability of the procedure to various types of humanoid robots and walking.
