専門家デモンストレーションを活用した都市自動運転のための改良深層強化学習
Improved Deep Reinforcement Learning with Expert Demonstrations for Urban Autonomous Driving
強化学習と模倣学習を組み合わせ、専門家のデモンストレーションを優先経験再生で活用することで、都市交差点での縦方向車両制御を効率的かつ安全に学習する手法を提案した論文。
著者: Haochen Liu, Zhiyu Huang, Jingda Wu, Chen Lv
分類: cs.RO
原文アブストラクト
Learning-based approaches, such as reinforcement learning (RL) and imitation learning (IL), have indicated superiority over rule-based approaches in complex urban autonomous driving environments, showing great potential to make intelligent decisions. However, current RL and IL approaches still have their own drawbacks, such as low data efficiency for RL and poor generalization capability for IL. In light of this, this paper proposes a novel learning-based method that combines deep reinforcement learning and imitation learning from expert demonstrations, which is applied to longitudinal vehicle motion control in autonomous driving scenarios. Our proposed method employs the soft actor-critic and modifies the learning process of the policy network to incorporate both the goals of maximizing reward and imitating the expert. Moreover, an adaptive prioritized experience replay is designed to sample experience from both the agent's self-exploration and expert demonstration, in order to improve sample efficiency. The proposed method is validated in a simulated urban roundabout scenario and compared with various prevailing RL and IL baselines. The results manifest that the proposed method has a faster training speed, as well as better performance in navigating safely and time-efficiently.
