中断を学習する階層型深層強化学習フレームワークによる効率的探索
Learning to Interrupt: A Hierarchical Deep Reinforcement Learning Framework for Efficient Exploration
人間設計のオプションと学習可能な終了関数による中断機構を組み合わせた階層型強化学習を提案し、四部屋ナビゲーションで効率性と柔軟性を示した。
著者: Tingguang Li, Jin Pan, Delong Zhu, Max Q. -H. Meng
分類: cs.AI, cs.RO
原文アブストラクト
To achieve scenario intelligence, humans must transfer knowledge to robots by developing goal-oriented algorithms, which are sometimes insensitive to dynamically changing environments. While deep reinforcement learning achieves significant success recently, it is still extremely difficult to be deployed in real robots directly. In this paper, we propose a hybrid structure named Option-Interruption in which human knowledge is embedded into a hierarchical reinforcement learning framework. Our architecture has two key components: options, represented by existing human-designed methods, can significantly speed up the training process and interruption mechanism, based on learnable termination functions, enables our system to quickly respond to the external environment. To implement this architecture, we derive a set of update rules based on policy gradient methods and present a complete training process. In the experiment part, our method is evaluated in Four-room navigation and exploration task, which shows the efficiency and flexibility of our framework.
