目標認識型クロスエントロピーによるマルチターゲット強化学習
Goal-Aware Cross-Entropy for Multi-Target Reinforcement Learning
複数目標環境での強化学習のため、自動ラベル付けした目標状態を用いる自己教師ありの目標認識型クロスエントロピー損失と、指示に注目する目標識別注意ネットワークを提案し、視覚ナビゲーションとロボットアーム操作で成功率・サンプル効率・汎化性能を向上させた。
著者: Kibeom Kim, Min Whoo Lee, Yoonsung Kim, Je-Hwan Ryu, Minsu Lee, Byoung-Tak Zhang
分類: cs.LG, cs.AI, cs.RO
原文アブストラクト
Learning in a multi-target environment without prior knowledge about the targets requires a large amount of samples and makes generalization difficult. To solve this problem, it is important to be able to discriminate targets through semantic understanding. In this paper, we propose goal-aware cross-entropy (GACE) loss, that can be utilized in a self-supervised way using auto-labeled goal states alongside reinforcement learning. Based on the loss, we then devise goal-discriminative attention networks (GDAN) which utilize the goal-relevant information to focus on the given instruction. We evaluate the proposed methods on visual navigation and robot arm manipulation tasks with multi-target environments and show that GDAN outperforms the state-of-the-art methods in terms of task success ratio, sample efficiency, and generalization. Additionally, qualitative analyses demonstrate that our proposed method can help the agent become aware of and focus on the given instruction clearly, promoting goal-directed behavior.
