安全なモデルベース能動強化学習フレームワークSAMBA
SAMBA: Safe Model-Based & Active Reinforcement Learning
確率モデリングと情報理論を組み合わせ、PILCOを拡張した安全制約付き能動探索により、サンプル数と違反を大幅削減する強化学習手法を提案した。
著者: Alexander I. Cowen-Rivers, Daniel Palenicek, Vincent Moens, Mohammed Abdullah, Aivar Sootla, Jun Wang, Haitham Ammar
分類: cs.LG, cs.AI, cs.RO, stat.ML
原文アブストラクト
In this paper, we propose SAMBA, a novel framework for safe reinforcement learning that combines aspects from probabilistic modelling, information theory, and statistics. Our method builds upon PILCO to enable active exploration using novel(semi-)metrics for out-of-sample Gaussian process evaluation optimised through a multi-objective problem that supports conditional-value-at-risk constraints. We evaluate our algorithm on a variety of safe dynamical system benchmarks involving both low and high-dimensional state representations. Our results show orders of magnitude reductions in samples and violations compared to state-of-the-art methods. Lastly, we provide intuition as to the effectiveness of the framework by a detailed analysis of our active metrics and safety constraints.
