挑戦的環境でのロボット探索のためのオンライン適応付きオフポリシー評価
Off-Policy Evaluation with Online Adaptation for Robot Exploration in Challenging Environments
実世界データで状態価値関数を学習し、TD法によるオンライン適応でロボット探索を誘導する手法を提案。地下や都市環境での探索性能を向上させた。
著者: Yafei Hu, Junyi Geng, Chen Wang, John Keller, Sebastian Scherer
分類: cs.RO, cs.AI, cs.LG
原文アブストラクト
Autonomous exploration has many important applications. However, classic information gain-based or frontier-based exploration only relies on the robot current state to determine the immediate exploration goal, which lacks the capability of predicting the value of future states and thus leads to inefficient exploration decisions. This paper presents a method to learn how "good" states are, measured by the state value function, to provide a guidance for robot exploration in real-world challenging environments. We formulate our work as an off-policy evaluation (OPE) problem for robot exploration (OPERE). It consists of offline Monte-Carlo training on real-world data and performs Temporal Difference (TD) online adaptation to optimize the trained value estimator. We also design an intrinsic reward function based on sensor information coverage to enable the robot to gain more information with sparse extrinsic rewards. Results show that our method enables the robot to predict the value of future states so as to better guide robot exploration. The proposed algorithm achieves better prediction and exploration performance compared with the state-of-the-arts. To the best of our knowledge, this work for the first time demonstrates value function prediction on real-world dataset for robot exploration in challenging subterranean and urban environments. More details and demo videos can be found at https://jeffreyyh.github.io/opere/.
