オフポリシー方策評価の実証ベンチマーク
Empirical Study of Off-Policy Policy Evaluation for Reinforcement Learning
強化学習におけるオフポリシー方策評価(OPE)手法を多様な実験設計で比較するベンチマークスイートCOBSを構築し、実践的な指針をまとめた。
著者: Cameron Voloshin, Hoang M. Le, Nan Jiang, Yisong Yue
分類: cs.LG, cs.AI, cs.RO, stat.ML
原文アブストラクト
We offer an experimental benchmark and empirical study for off-policy policy evaluation (OPE) in reinforcement learning, which is a key problem in many safety critical applications. Given the increasing interest in deploying learning-based methods, there has been a flurry of recent proposals for OPE method, leading to a need for standardized empirical analyses. Our work takes a strong focus on diversity of experimental design to enable stress testing of OPE methods. We provide a comprehensive benchmarking suite to study the interplay of different attributes on method performance. We distill the results into a summarized set of guidelines for OPE in practice. Our software package, the Caltech OPE Benchmarking Suite (COBS), is open-sourced and we invite interested researchers to further contribute to the benchmark.
関連論文
- 深層強化学習評価におけるタスクの不完全指定の影響強化学習評価
