高コンフリクトな運転場面における双方向レーン利用のロバストな交渉学習
Learning to Robustly Negotiate Bi-Directional Lane Usage in High-Conflict Driving Scenarios
対向車と優先権が等しく通信も集中制御もない日常的な高コンフリクト場面で、相手の協力度が未知でも交渉できるようMARLアルゴリズムDASACを提案し、99%以上の成功率と人間らしい防御運転を実現した。
著者: Christoph Killing, Adam Villaflor, John M. Dolan
分類: cs.LG, cs.AI, cs.MA, cs.RO
原文アブストラクト
Recently, autonomous driving has made substantial progress in addressing the most common traffic scenarios like intersection navigation and lane changing. However, most of these successes have been limited to scenarios with well-defined traffic rules and require minimal negotiation with other vehicles. In this paper, we introduce a previously unconsidered, yet everyday, high-conflict driving scenario requiring negotiations between agents of equal rights and priorities. There exists no centralized control structure and we do not allow communications. Therefore, it is unknown if other drivers are willing to cooperate, and if so to what extent. We train policies to robustly negotiate with opposing vehicles of an unobservable degree of cooperativeness using multi-agent reinforcement learning (MARL). We propose Discrete Asymmetric Soft Actor-Critic (DASAC), a maximum-entropy off-policy MARL algorithm allowing for centralized training with decentralized execution. We show that using DASAC we are able to successfully negotiate and traverse the scenario considered over 99% of the time. Our agents are robust to an unknown timing of opponent decisions, an unobservable degree of cooperativeness of the opposing vehicle, and previously unencountered policies. Furthermore, they learn to exhibit human-like behaviors such as defensive driving, anticipating solution options and interpreting the behavior of other agents.
