DHRL: グラフを用いた長期的・スパースな階層強化学習
DHRL: A Graph-Based Approach for Long-Horizon and Sparse Hierarchical Reinforcement Learning
階層強化学習において、高レベルと低レベルのポリシーの時間軸をグラフで分離し、長期的なタスクを効率的に学習する手法を提案。
著者: Seungjae Lee, Jigang Kim, Inkyu Jang, H. Jin Kim
分類: cs.LG, cs.AI
原文アブストラクト
Hierarchical Reinforcement Learning (HRL) has made notable progress in complex control tasks by leveraging temporal abstraction. However, previous HRL algorithms often suffer from serious data inefficiency as environments get large. The extended components, $i.e.$, goal space and length of episodes, impose a burden on either one or both high-level and low-level policies since both levels share the total horizon of the episode. In this paper, we present a method of Decoupling Horizons Using a Graph in Hierarchical Reinforcement Learning (DHRL) which can alleviate this problem by decoupling the horizons of high-level and low-level policies and bridging the gap between the length of both horizons using a graph. DHRL provides a freely stretchable high-level action interval, which facilitates longer temporal abstraction and faster training in complex tasks. Our method outperforms state-of-the-art HRL algorithms in typical HRL environments. Moreover, DHRL achieves long and complex locomotion and manipulation tasks.
