到達可能性解析による階層強化学習の目標空間抽象化
Goal Space Abstraction in Hierarchical Reinforcement Learning via Reachability Analysis
環境状態をタスク上の役割が似たグループに抽象化する表現を自律的に獲得し、方策と同時に学習する階層強化学習アルゴリズムを提案。ナビゲーションタスクで解釈性とデータ効率を実証した。
著者: Mehdi Zadem, Sergio Mover, Sao Mai Nguyen
分類: cs.LG, cs.AI, cs.FL, cs.RO
原文アブストラクト
Open-ended learning benefits immensely from the use of symbolic methods for goal representation as they offer ways to structure knowledge for efficient and transferable learning. However, the existing Hierarchical Reinforcement Learning (HRL) approaches relying on symbolic reasoning are often limited as they require a manual goal representation. The challenge in autonomously discovering a symbolic goal representation is that it must preserve critical information, such as the environment dynamics. In this work, we propose a developmental mechanism for subgoal discovery via an emergent representation that abstracts (i.e., groups together) sets of environment states that have similar roles in the task. We create a HRL algorithm that gradually learns this representation along with the policies and evaluate it on navigation tasks to show the learned representation is interpretable and results in data efficiency.
