好奇心駆動の階層型アクター・クリティック強化学習
Curious Hierarchical Actor-Critic Reinforcement Learning
階層型強化学習に順モデルによる好奇心を組み合わせ、連続空間環境で学習性能と成功率が最大2倍以上に向上することを示した研究。
著者: Frank Röder, Manfred Eppe, Phuong D. H. Nguyen, Stefan Wermter
分類: cs.LG, cs.RO, stat.ML
原文アブストラクト
Hierarchical abstraction and curiosity-driven exploration are two common paradigms in current reinforcement learning approaches to break down difficult problems into a sequence of simpler ones and to overcome reward sparsity. However, there is a lack of approaches that combine these paradigms, and it is currently unknown whether curiosity also helps to perform the hierarchical abstraction. As a novelty and scientific contribution, we tackle this issue and develop a method that combines hierarchical reinforcement learning with curiosity. Herein, we extend a contemporary hierarchical actor-critic approach with a forward model to develop a hierarchical notion of curiosity. We demonstrate in several continuous-space environments that curiosity can more than double the learning performance and success rates for most of the investigated benchmarking problems. We also provide our source code and a supplementary video.
