強化学習の報酬形成のための時間的動画言語アラインメントネットワーク
Temporal Video-Language Alignment Network for Reward Shaping in Reinforcement Learning
自然言語指示を中間報酬に変換する報酬形成手法を提案し、AtariのMontezuma's Revengeで従来手法より高いタスク達成率を実現した。
著者: Ziyuan Cao, Reshma Anugundanahalli Ramachandra, Kelin Yu
分類: cs.RO
原文アブストラクト
Designing appropriate reward functions for Reinforcement Learning (RL) approaches has been a significant problem, especially for complex environments such as Atari games. Utilizing natural language instructions to provide intermediate rewards to RL agents in a process known as reward shaping can help the agent in reaching the goal state faster. In this work, we propose a natural language-based reward shaping approach that maps trajectories from the Montezuma's Revenge game environment to corresponding natural language instructions using an extension of the LanguagE-Action Reward Network (LEARN) framework. These trajectory-language mappings are further used to generate intermediate rewards which are integrated into reward functions that can be utilized to learn an optimal policy for any standard RL algorithms. For a set of 15 tasks from Atari's Montezuma's Revenge game, the Ext-LEARN approach leads to the successful completion of tasks more often on average than the reward shaping approach that uses the LEARN framework and performs even better than the reward shaping framework without natural language-based rewards.
