AlphaDrive:強化学習と推論で自動運転におけるVLMの能力を引き出す
AlphaDrive: Unleashing the Power of VLMs in Autonomous Driving via Reinforcement Learning and Reasoning
自動運転向けにGRPOベースの強化学習報酬と2段階の計画推論訓練を組み合わせたVLMフレームワークを提案し、計画性能と訓練効率を改善した。
著者: Bo Jiang, Shaoyu Chen, Qian Zhang, Wenyu Liu, Xinggang Wang
分類: cs.CV, cs.RO
原文アブストラクト
OpenAI o1 and DeepSeek R1 achieve or even surpass human expert-level performance in complex domains like mathematics and science, with reinforcement learning (RL) and reasoning playing a crucial role. In autonomous driving, recent end-to-end models have greatly improved planning performance but still struggle with long-tailed problems due to limited common sense and reasoning abilities. Some studies integrate vision-language models (VLMs) into autonomous driving, but they typically rely on pre-trained models with simple supervised fine-tuning (SFT) on driving data, without further exploration of training strategies or optimizations specifically tailored for planning. In this paper, we propose AlphaDrive, a RL and reasoning framework for VLMs in autonomous driving. AlphaDrive introduces four GRPO-based RL rewards tailored for planning and employs a two-stage planning reasoning training strategy that combines SFT with RL. As a result, AlphaDrive significantly improves both planning performance and training efficiency compared to using only SFT or without reasoning. Moreover, we are also excited to discover that, following RL training, AlphaDrive exhibits some emergent multimodal planning capabilities, which is critical for improving driving safety and efficiency. To the best of our knowledge, AlphaDrive is the first to integrate GRPO-based RL with planning reasoning into autonomous driving. Code will be released to facilitate future research.
関連論文
- 視覚マスクと緊急ブレーキを備えたGatedVLA:AlpaSim閉ループ走行チャレンジ技術報告自動運転VLA
- STEER: 経路認識型適応推論による自動運転自動運転VLA
- SafeDriveVLA: ナビゲーション条件付きワールドモデルドリーミングによる衝突認識型エンドツーエンド自動運転自動運転VLA
- 走ってから歩け:VLM自動運転のためのRun-then-Walkスケジューリング戦略自動運転VLA
- AppleVLM:高度な知覚と計画を強化した視覚言語モデルによるエンドツーエンド自動運転自動運転VLA
- SToRM: マルチモーダルLLMのための教師ありトークン削減による効率的なエンドツーエンド自動運転自動運転VLA
