安全指数ガイド付きガウス過程モデルによる強化学習の確率的セーフガード
Probabilistic Safeguard for Reinforcement Learning Using Safety Index Guided Gaussian Process Models
強化学習エージェントの動力学をガウス過程で学習し、安全指数に基づく安全制御と組み合わせることで、ハードな状態制約をほぼ違反せずに満たす枠組みを提案した。
著者: Weiye Zhao, Tairan He, Changliu Liu
分類: cs.RO
原文アブストラクト
Safety is one of the biggest concerns to applying reinforcement learning (RL) to the physical world. In its core part, it is challenging to ensure RL agents persistently satisfy a hard state constraint without white-box or black-box dynamics models. This paper presents an integrated model learning and safe control framework to safeguard any agent, where its dynamics are learned as Gaussian processes. The proposed theory provides (i) a novel method to construct an offline dataset for model learning that best achieves safety requirements; (ii) a parameterization rule for safety index to ensure the existence of safe control; (iii) a safety guarantee in terms of probabilistic forward invariance when the model is learned using the aforementioned dataset. Simulation results show that our framework guarantees almost zero safety violation on various continuous control tasks.
