移動マニピュレータのための対話型ナビゲーション階層強化学習
HRL4IN: Hierarchical Reinforcement Learning for Interactive Navigation with Mobile Manipulators
ドア開けやボタン押しを伴う対話型ナビゲーションを、フェーズごとに異なる空間でサブゴールを設定し身体部位を使い分ける階層強化学習HRL4INで解き、平坦なPPOやHACより高性能かつ省エネルギーであることを示した。
著者: Chengshu Li, Fei Xia, Roberto Martin-Martin, Silvio Savarese
分類: cs.LG, cs.AI, cs.CV, cs.RO, stat.ML
原文アブストラクト
Most common navigation tasks in human environments require auxiliary arm interactions, e.g. opening doors, pressing buttons and pushing obstacles away. This type of navigation tasks, which we call Interactive Navigation, requires the use of mobile manipulators: mobile bases with manipulation capabilities. Interactive Navigation tasks are usually long-horizon and composed of heterogeneous phases of pure navigation, pure manipulation, and their combination. Using the wrong part of the embodiment is inefficient and hinders progress. We propose HRL4IN, a novel Hierarchical RL architecture for Interactive Navigation tasks. HRL4IN exploits the exploration benefits of HRL over flat RL for long-horizon tasks thanks to temporally extended commitments towards subgoals. Different from other HRL solutions, HRL4IN handles the heterogeneous nature of the Interactive Navigation task by creating subgoals in different spaces in different phases of the task. Moreover, HRL4IN selects different parts of the embodiment to use for each phase, improving energy efficiency. We evaluate HRL4IN against flat PPO and HAC, a state-of-the-art HRL algorithm, on Interactive Navigation in two environments - a 2D grid-world environment and a 3D environment with physics simulation. We show that HRL4IN significantly outperforms its baselines in terms of task performance and energy efficiency. More information is available at https://sites.google.com/view/hrl4in.
