Safe-RULE: 安全な強化学習のアンラーニング
Safe-RULE: Safe Reinforcement UnLEarning
オフライン安全強化学習に対するデータポイズニング攻撃の影響を除去するための防御フレームワークを提案し、再学習や元の環境へのアクセスなしに安全性を回復する手法を実証した。
詳しい要約
1. どんなもの?
2. 先行研究と比べてどこがすごい?
3. 技術・手法の肝は?
4. どうやって有効だと検証した?
5. 議論はある?
6. 次に読むべき論文は?
※ AIが要旨から生成した要約です。正確性は原文をご確認ください。
著者: Shixiong Jiang, Taozheng Zhu, Fanxin Kong
分類: cs.LG, cs.AI, cs.CR, cs.RO
原文アブストラクト
Offline safe reinforcement learning (Safe RL) enables policy learning without online interactions, making it suitable for safety-critical systems such as robotics systems. However, its reliance on static datasets exposes offline Safe RL to data poisoning attacks, where adversaries inject malicious samples that compromise safety and induce unsafe policy behavior. In this work, we propose a new learning paradigm, named safe reinforcement unlearning (Safe-RULE), used as a defense framework to remove the influence of poisoned data without retraining from scratch or requiring access to the original training environment. We further extend reinforcement unlearning to offline Safe RL by explicitly accounting for both task performance and safety constraints during the unlearning process. Experiments across benchmark Safe RL tasks demonstrate that our approach effectively enhances safety performance against data poisoning attacks.
