動的システムの安全強化学習のための安全領域の低次元表現の学習
Learning a Low-dimensional Representation of a Safe Region for Safe Reinforcement Learning on Dynamical Systems
複雑な動的システムでも人手設計なしにデータから安全領域の低次元表現を学習し、オンライン更新で安全推定を高精度化する手法を提案し、クアッドコプターで有効性を示した。
著者: Zhehua Zhou, Ozgur S. Oguz, Marion Leibold, Martin Buss
分類: cs.RO, cs.SY, eess.SY
原文アブストラクト
For safely applying reinforcement learning algorithms on high-dimensional nonlinear dynamical systems, a simplified system model is used to formulate a safe reinforcement learning framework. Based on the simplified system model, a low-dimensional representation of the safe region is identified and is used to provide safety estimates for learning algorithms. However, finding a satisfying simplified system model for complex dynamical systems usually requires a considerable amount of effort. To overcome this limitation, we propose in this work a general data-driven approach that is able to efficiently learn a low-dimensional representation of the safe region. Through an online adaptation method, the low-dimensional representation is updated by using the feedback data such that more accurate safety estimates are obtained. The performance of the proposed approach for identifying the low-dimensional representation of the safe region is demonstrated with a quadcopter example. The results show that, compared to previous work, a more reliable and representative low-dimensional representation of the safe region is derived, which then extends the applicability of the safe reinforcement learning framework.
