見るものを減らして運転を改善：基盤モデルの確率的パッチ選択による汎化可能なEnd-to-End自動運転
See Less, Drive Better: Generalizable End-to-End Autonomous Driving via Foundation Models Stochastic Patch Selection
基盤モデルから抽出したパッチ特徴の冗長性を削減するため、ランダムに一部のパッチをマスクして学習するStochastic-Patch-Selectionを提案し、分布外シナリオでの自動運転性能を平均6.2%向上させた。
詳しい要約
1. どんなもの？
2. 先行研究と比べてどこがすごい？
3. 技術・手法の肝は？
4. どうやって有効だと検証した？
5. 議論はある？
6. 次に読むべき論文は？
※ AIが要旨から生成した要約です。正確性は原文をご確認ください。
著者: Amir Mallak, Erfan Aasi, Shiva Sreeram, Tsun-Hsuan Wang, Daniela Rus, Alaa Maalouf
分類: cs.CV, cs.LG, cs.RO
原文アブストラクト
Recent advances in end-to-end autonomous driving show that policies trained on patch-aligned features extracted from foundation models generalize better to Out-of-Distribution (OOD). We hypothesize that due to the self-attention mechanism, each patch feature implicitly embeds/contains information from all other patches, represented in a different way and intensity, making these descriptors highly redundant. We quantify redundancy in such (BLIP2) features via PCA and cross-patch similarity: $90$% of variance is captured by $17/64$ principal components, and strong inter-token correlations are pervasive. Training on such overlapping information leads the policy to overfit spurious correlations, hurting OOD robustness. We present Stochastic-Patch-Selection (SPS), a simple yet effective approach for learning policies that are more robust, generalizable, and efficient. For every frame, SPS randomly masks a fraction of patch descriptors, not feeding them to the policy model, while preserving the spatial layout of the remaining patches. Thus, the policy is provided with different stochastic but complete views of the (same) scene: every random subset of patches acts like a different, yet still sensible, coherent projection of the world. The policy thus bases its decisions on features that are invariant to which specific tokens survive. Extensive experiments confirm that across all OOD scenarios, our method outperforms the state of the art (SOTA), achieving a $6.2$% average improvement and up to $20.4$% in closed-loop simulations, while being $2.4\times$ faster. We conduct ablations over masking rates and patch-feature reorganization, training and evaluating 9 systems, with 8 of them surpassing prior SOTA. Finally, we show that the same learned policy transfers to a physical, real-world car without any tuning.
関連論文
- 蒸留サロゲートで安全クリティカルなシーンを生成し堅牢なEnd-to-End自動運転を実現End-to-End自動運転
- SC-IMMベースの教師信号によるEnd-to-End自動運転の軌道出力安定化End-to-End自動運転
- 語彙候補と拡散によるシーン適応的洗練を統合したEnd-to-End自動運転End-to-End自動運転
- Raw2Drive: 世界モデル整合による強化学習ベースのEnd-to-End自動運転End-to-End自動運転
- EgoFSD: 不確実性デノイジングと反復的洗練による効率的なEnd-to-End自動運転のための自己中心的全スパースパラダイムEnd-to-End自動運転
- 模倣学習によるEnd-to-End自動運転の交通ルール遵守を高める要因End-to-End自動運転
