目標関係グラフを用いた階層型部分観測目標駆動方策学習
Hierarchical and Partially Observable Goal-driven Policy Learning with Goals Relational Graph
目標空間内の関係をディリクレ・カテゴリカル過程で捉える目標関係グラフを導入し、高レベルがサブゴールを生成、低レベルが最適方策を学習する二層階層強化学習を提案。未見環境や新規目標への汎化性能をグリッドワールドとロボット物体探索で示した。
著者: Xin Ye, Yezhou Yang
分類: cs.CV, cs.RO
原文アブストラクト
We present a novel two-layer hierarchical reinforcement learning approach equipped with a Goals Relational Graph (GRG) for tackling the partially observable goal-driven task, such as goal-driven visual navigation. Our GRG captures the underlying relations of all goals in the goal space through a Dirichlet-categorical process that facilitates: 1) the high-level network raising a sub-goal towards achieving a designated final goal; 2) the low-level network towards an optimal policy; and 3) the overall system generalizing unseen environments and goals. We evaluate our approach with two settings of partially observable goal-driven tasks -- a grid-world domain and a robotic object search task. Our experimental results show that our approach exhibits superior generalization performance on both unseen environments and new goals.
