関数近似による大規模状態空間での逆強化学習
Inverse Reinforcement Learning in Large State Spaces via Function Approximation
ベルマン最適方程式を常に満たす関数近似を用いて、大規模・高次元状態空間でも効率的に逆強化学習を行う手法を提案し、脊髄損傷患者のリハビリ刺激効果評価にも応用した。
著者: Kun Li, Joel W. Burdick
分類: cs.LG, cs.RO
原文アブストラクト
This paper introduces a new method for inverse reinforcement learning in large-scale and high-dimensional state spaces. To avoid solving the computationally expensive reinforcement learning problems in reward learning, we propose a function approximation method to ensure that the Bellman Optimality Equation always holds, and then estimate a function to maximize the likelihood of the observed motion. The time complexity of the proposed method is linearly proportional to the cardinality of the action set, thus it can handle large state spaces efficiently. We test the proposed method in a simulated environment, and show that it is more accurate than existing methods and significantly better in scalability. We also show that the proposed method can extend many existing methods to high-dimensional state spaces. We then apply the method to evaluating the effect of rehabilitative stimulations on patients with spinal cord injuries based on the observed patient motions.
