最大事後確率方策最適化
Maximum a Posteriori Policy Optimisation
相対エントロピー目的関数の座標上昇に基づく新しい強化学習アルゴリズムMPOを提案し、連続制御でサンプル効率と安定性に優れることを示した。
著者: Abbas Abdolmaleki, Jost Tobias Springenberg, Yuval Tassa, Remi Munos, Nicolas Heess, Martin Riedmiller
分類: cs.LG, cs.AI, cs.IT, cs.RO, math.IT, stat.ML
原文アブストラクト
We introduce a new algorithm for reinforcement learning called Maximum aposteriori Policy Optimisation (MPO) based on coordinate ascent on a relative entropy objective. We show that several existing methods can directly be related to our derivation. We develop two off-policy algorithms and demonstrate that they are competitive with the state-of-the-art in deep reinforcement learning. In particular, for continuous control, our method outperforms existing methods with respect to sample efficiency, premature convergence and robustness to hyperparameter settings while achieving similar or better final performance.
