分散型複数制御バリア関数による安全なマルチエージェント強化学習
Safe Multi-Agent Reinforcement Learning through Decentralized Multiple Control Barrier Functions
集中型シールドの代わりに、局所情報のみで動作する分散型の複数制御バリア関数(CBF)シールドをMARLに組み込み、MADDPGを拡張したMADDPG-CBFを提案した論文。衝突回避問題で理論的な安全保証と実験による安全性の検証を示している。
著者: Zhiyuan Cai, Huanhui Cao, Wenjie Lu, Lin Zhang, Hao Xiong
分類: cs.MA, cs.RO
原文アブストラクト
Multi-Agent Reinforcement Learning (MARL) algorithms show amazing performance in simulation in recent years, but placing MARL in real-world applications may suffer safety problems. MARL with centralized shields was proposed and verified in safety games recently. However, centralized shielding approaches can be infeasible in several real-world multi-agent applications that involve non-cooperative agents or communication delay. Thus, we propose to combine MARL with decentralized Control Barrier Function (CBF) shields based on available local information. We establish a safe MARL framework with decentralized multiple CBFs and develop Multi-Agent Deep Deterministic Policy Gradient (MADDPG) to Multi-Agent Deep Deterministic Policy Gradient with decentralized multiple Control Barrier Functions (MADDPG-CBF). Based on a collision-avoidance problem that includes not only cooperative agents but obstacles, we demonstrate the construction of multiple CBFs with safety guarantees in theory. Experiments are conducted and experiment results verify that the proposed safe MARL framework can guarantee the safety of agents included in MARL.
