階層強化学習による汎化可能な移動スキルの学習
Learning Generalizable Locomotion Skills with Hierarchical Reinforcement Learning
移動を多様なプリミティブスキルの学習とモデルベース計画による系列化に分割する階層学習フレームワークを提案し、実機の6脚ロボットで約2時間の学習から12m先の目標まで歩行できることを示した。
著者: Tianyu Li, Nathan Lambert, Roberto Calandra, Franziska Meier, Akshara Rai
分類: cs.RO
原文アブストラクト
Learning to locomote to arbitrary goals on hardware remains a challenging problem for reinforcement learning. In this paper, we present a hierarchical learning framework that improves sample-efficiency and generalizability of locomotion skills on real-world robots. Our approach divides the problem of goal-oriented locomotion into two sub-problems: learning diverse primitives skills, and using model-based planning to sequence these skills. We parametrize our primitives as cyclic movements, improving sample-efficiency of learning on a 18 degrees of freedom robot. Then, we learn coarse dynamics models over primitive cycles and use them in a model predictive control framework. This allows us to learn to walk to arbitrary goals up to 12m away, after about two hours of training from scratch on hardware. Our results on a Daisy hexapod hardware and simulation demonstrate the efficacy of our approach at reaching distant targets, in different environments and with sensory noise.
