階層型強化学習による複雑な交通シナリオでの広範な探索
Extensive Exploration in Complex Traffic Scenarios using Hierarchical Reinforcement Learning
複雑な交通環境での自動運転制御に向け、意思決定を階層的に分解し高レベル・低レベル制御器を別々に訓練する強化学習フレームワークを提案し、複雑な高速道路状況での優位性を示した。
著者: Zhihao Zhang, Ekim Yurtsever, Keith A. Redmill
分類: cs.LG, cs.RO
原文アブストラクト
Developing an automated driving system capable of navigating complex traffic environments remains a formidable challenge. Unlike rule-based or supervised learning-based methods, Deep Reinforcement Learning (DRL) based controllers eliminate the need for domain-specific knowledge and datasets, thus providing adaptability to various scenarios. Nonetheless, a common limitation of existing studies on DRL-based controllers is their focus on driving scenarios with simple traffic patterns, which hinders their capability to effectively handle complex driving environments with delayed, long-term rewards, thus compromising the generalizability of their findings. In response to these limitations, our research introduces a pioneering hierarchical framework that efficiently decomposes intricate decision-making problems into manageable and interpretable subtasks. We adopt a two step training process that trains the high-level controller and low-level controller separately. The high-level controller exhibits an enhanced exploration potential with long-term delayed rewards, and the low-level controller provides longitudinal and lateral control ability using short-term instantaneous rewards. Through simulation experiments, we demonstrate the superiority of our hierarchical controller in managing complex highway driving situations.
