記号的な歩行記述から強化学習の報酬を導出する手法
Deriving Rewards for Reinforcement Learning from Symbolic Behaviour Descriptions of Bipedal Walking
二足歩行の記号的な振る舞い記述をハイブリッドオートマトンとしてモデル化し、そこから強化学習の報酬関数を自動生成することで、学習時間を短縮しつつ歩行速度を向上させた。
著者: Daniel Harnack, Christoph Lüth, Lukas Gross, Shivesh Kumar, Frank Kirchner
分類: cs.RO, cs.LG, cs.LO
原文アブストラクト
Generating physical movement behaviours from their symbolic description is a long-standing challenge in artificial intelligence (AI) and robotics, requiring insights into numerical optimization methods as well as into formalizations from symbolic AI and reasoning. In this paper, a novel approach to finding a reward function from a symbolic description is proposed. The intended system behaviour is modelled as a hybrid automaton, which reduces the system state space to allow more efficient reinforcement learning. The approach is applied to bipedal walking, by modelling the walking robot as a hybrid automaton over state space orthants, and used with the compass walker to derive a reward that incentivizes following the hybrid automaton cycle. As a result, training times of reinforcement learning controllers are reduced while final walking speed is increased. The approach can serve as a blueprint how to generate reward functions from symbolic AI and reasoning.
