安全介入を減らす証明可能な安全強化学習
Reducing Safety Interventions in Provably Safe Reinforcement Learning
強化学習エージェントの行動がフェイルセーフ介入を引き起こしそうな場合に事前に行動を置換・投影することで、安全性を保ちつつ介入回数を削減する手法を提案し、安全ジムベンチマークと人間-ロボット協調タスクで有効性を示した。
著者: Jakob Thumm, Guillaume Pelat, Matthias Althoff
分類: cs.RO
原文アブストラクト
Deep Reinforcement Learning (RL) has shown promise in addressing complex robotic challenges. In real-world applications, RL is often accompanied by failsafe controllers as a last resort to avoid catastrophic events. While necessary for safety, these interventions can result in undesirable behaviors, such as abrupt braking or aggressive steering. This paper proposes two safety intervention reduction methods: proactive replacement and proactive projection, which change the action of the agent if it leads to a potential failsafe intervention. These approaches are compared to state-of-the-art constrained RL on the OpenAI safety gym benchmark and a human-robot collaboration task. Our study demonstrates that the combination of our method with provably safe RL leads to high-performing policies with zero safety violations and a low number of failsafe interventions. Our versatile method can be applied to a wide range of real-world robotic tasks, while effectively improving safety without sacrificing task performance.
