ロバスト制御バリア関数を用いた安全な強化学習
Safe Reinforcement Learning Using Robust Control Barrier Functions
モデルベース強化学習に微分可能なロバスト制御バリア関数レイヤーを組み込み、報酬駆動タスクを安全制約と独立にモジュール学習することで、学習中の安全性と探索誘導を両立させた論文。
著者: Yousef Emam, Gennaro Notomista, Paul Glotfelter, Zsolt Kira, Magnus Egerstedt
分類: eess.SY, cs.AI, cs.LG, cs.RO, cs.SY
原文アブストラクト
Reinforcement Learning (RL) has been shown to be effective in many scenarios. However, it typically requires the exploration of a sufficiently large number of state-action pairs, some of which may be unsafe. Consequently, its application to safety-critical systems remains a challenge. An increasingly common approach to address safety involves the addition of a safety layer that projects the RL actions onto a safe set of actions. In turn, a difficulty for such frameworks is how to effectively couple RL with the safety layer to improve the learning performance. In this paper, we frame safety as a differentiable robust-control-barrier-function layer in a model-based RL framework. Moreover, we also propose an approach to modularly learn the underlying reward-driven task, independent of safety constraints. We demonstrate that this approach both ensures safety and effectively guides exploration during training in a range of experiments, including zero-shot transfer when the reward is learned in a modular way.
