データ駆動型予測制御を用いた安全な強化学習
Safe Reinforcement Learning using Data-Driven Predictive Control
強化学習の安全性を確保するため、データ駆動型予測制御による安全フィルタを提案し、ロボットナビゲーションで既存手法を上回る性能を示した。
著者: Mahmoud Selim, Amr Alanwar, M. Watheq El-Kharashi, Hazem M. Abbas, Karl H. Johansson
分類: cs.RO, cs.LG, cs.SY, eess.SY
原文アブストラクト
Reinforcement learning (RL) algorithms can achieve state-of-the-art performance in decision-making and continuous control tasks. However, applying RL algorithms on safety-critical systems still needs to be well justified due to the exploration nature of many RL algorithms, especially when the model of the robot and the environment are unknown. To address this challenge, we propose a data-driven safety layer that acts as a filter for unsafe actions. The safety layer uses a data-driven predictive controller to enforce safety guarantees for RL policies during training and after deployment. The RL agent proposes an action that is verified by computing the data-driven reachability analysis. If there is an intersection between the reachable set of the robot using the proposed action, we call the data-driven predictive controller to find the closest safe action to the proposed unsafe action. The safety layer penalizes the RL agent if the proposed action is unsafe and replaces it with the closest safe one. In the simulation, we show that our method outperforms state-of-the-art safe RL methods on the robotics navigation problem for a Turtlebot 3 in Gazebo and a quadrotor in Unreal Engine 4 (UE4).
