価値関数は制御バリア関数である：制御理論による安全な方策の検証
Value Functions are Control Barrier Functions: Verification of Safe Policies using Control Theory
強化学習の方策の安全性を保証するため、学習された価値関数と制御バリア関数の理論的関連を明らかにし、制御理論の検証手法を適用する新しい枠組みを提案した。
著者: Daniel C. H. Tan, Fernando Acero, Robert McCarthy, Dimitrios Kanoulas, Zhibin Li
分類: cs.LG, cs.AI, cs.RO
原文アブストラクト
Guaranteeing safe behaviour of reinforcement learning (RL) policies poses significant challenges for safety-critical applications, despite RL's generality and scalability. To address this, we propose a new approach to apply verification methods from control theory to learned value functions. By analyzing task structures for safety preservation, we formalize original theorems that establish links between value functions and control barrier functions. Further, we propose novel metrics for verifying value functions in safe control tasks and practical implementation details to improve learning. Our work presents a novel method for certificate learning, which unlocks a diversity of verification techniques from control theory for RL policies, and marks a significant step towards a formal framework for the general, scalable, and verifiable design of RL-based control systems. Code and videos are available at this https url: https://rl-cbf.github.io/
