累積カリキュラム強化学習による移動ロボットの自律探索と地図構築
Autonomous Exploration and Mapping for Mobile Robots via Cumulative Curriculum Reinforcement Learning
カリキュラム学習と深層強化学習を組み合わせ、破滅的忘却を抑えつつ未知サイズの環境に適応する移動ロボットの探索・地図構築手法を提案し、軽量シミュレータで効率を検証した。
著者: Zhi Li, Jinghao Xin, Ning Li
分類: cs.RO
原文アブストラクト
Deep reinforcement learning (DRL) has been widely applied in autonomous exploration and mapping tasks, but often struggles with the challenges of sampling efficiency, poor adaptability to unknown map sizes, and slow simulation speed. To speed up convergence, we combine curriculum learning (CL) with DRL, and first propose a Cumulative Curriculum Reinforcement Learning (CCRL) training framework to alleviate the issue of catastrophic forgetting faced by general CL. Besides, we present a novel state representation, which considers a local egocentric map and a global exploration map resized to the fixed dimension, so as to flexibly adapt to environments with various sizes and shapes. Additionally, for facilitating the fast training of DRL models, we develop a lightweight grid-based simulator, which can substantially accelerate simulation compared to popular robot simulation platforms such as Gazebo. Based on the customized simulator, comprehensive experiments have been conducted, and the results show that the CCRL framework not only mitigates the catastrophic forgetting problem, but also improves the sample efficiency and generalization of DRL models, compared to general CL as well as without a curriculum. Our code is available at https://github.com/BeamanLi/CCRL_Exploration.
