ベイズ世界モデルによる制約付き方策最適化
Constrained Policy Optimization via Bayesian World Models
ベイズ世界モデルの不確実性を利用し、タスク目的の楽観的上界と安全制約の悲観的上界を最大化するモデルベース強化学習手法LAMBDAを提案し、Safety-Gymで高いサンプル効率と制約違反の少なさを示した。
著者: Yarden As, Ilnura Usmanova, Sebastian Curi, Andreas Krause
分類: cs.LG, cs.AI, cs.RO
原文アブストラクト
Improving sample-efficiency and safety are crucial challenges when deploying reinforcement learning in high-stakes real world applications. We propose LAMBDA, a novel model-based approach for policy optimization in safety critical tasks modeled via constrained Markov decision processes. Our approach utilizes Bayesian world models, and harnesses the resulting uncertainty to maximize optimistic upper bounds on the task objective, as well as pessimistic upper bounds on the safety constraints. We demonstrate LAMBDA's state of the art performance on the Safety-Gym benchmark suite in terms of sample efficiency and constraint violation.
