安全強化学習における低次元安全領域学習のためのデータ生成法
Data Generation Method for Learning a Low-dimensional Safe Region in Safe Reinforcement Learning
安全強化学習で低次元の安全領域を学習する際、訓練データの生成方法が性能と安全性に与える影響を調べ、2つのサンプリング手法を組み合わせた代表的なデータ生成法を提案し、3リンク倒立振子で有効性を示した論文。
著者: Zhehua Zhou, Ozgur S. Oguz, Yi Ren, Marion Leibold, Martin Buss
分類: eess.SY, cs.LG, cs.RO, cs.SY
原文アブストラクト
Safe reinforcement learning aims to learn a control policy while ensuring that neither the system nor the environment gets damaged during the learning process. For implementing safe reinforcement learning on highly nonlinear and high-dimensional dynamical systems, one possible approach is to find a low-dimensional safe region via data-driven feature extraction methods, which provides safety estimates to the learning algorithm. As the reliability of the learned safety estimates is data-dependent, we investigate in this work how different training data will affect the safe reinforcement learning approach. By balancing between the learning performance and the risk of being unsafe, a data generation method that combines two sampling methods is proposed to generate representative training data. The performance of the method is demonstrated with a three-link inverted pendulum example.
