深層強化学習を実践で機能させる方法
How to Make Deep RL Work in Practice
深層強化学習の性能に影響する初期化・入力正規化・適応学習手法の効果を調査し、デフォルトで使うべき手法とRL特化の改善が必要な領域を提案した論文。
著者: Nirnai Rao, Elie Aljalbout, Axel Sauer, Sami Haddadin
分類: cs.LG, cs.RO
原文アブストラクト
In recent years, challenging control problems became solvable with deep reinforcement learning (RL). To be able to use RL for large-scale real-world applications, a certain degree of reliability in their performance is necessary. Reported results of state-of-the-art algorithms are often difficult to reproduce. One reason for this is that certain implementation details influence the performance significantly. Commonly, these details are not highlighted as important techniques to achieve state-of-the-art performance. Additionally, techniques from supervised learning are often used by default but influence the algorithms in a reinforcement learning setting in different and not well-understood ways. In this paper, we investigate the influence of certain initialization, input normalization, and adaptive learning techniques on the performance of state-of-the-art RL algorithms. We make suggestions which of those techniques to use by default and highlight areas that could benefit from a solution specifically tailored to RL.
