CRISP: 予測型ワールドモデル事前学習による運転用時空間カメラ・レーダーバックボーン
CRISP: A Spatiotemporal Camera-Radar Backbone for Driving via Forecasting-Based World-Model Pretraining
カメラとレーダーの融合表現を、将来のLiDAR点群予測という事前学習タスクで学習するバックボーンを提案。推論時はカメラとレーダーのみで動作し、下流タスクに転移可能。
詳しい要約
1. どんなもの?
2. 先行研究と比べてどこがすごい?
3. 技術・手法の肝は?
4. どうやって有効だと検証した?
5. 議論はある?
6. 次に読むべき論文は?
※ AIが要旨から生成した要約です。正確性は原文をご確認ください。
著者: Jingyu Song, Yi Liu, Katherine A. Skinner
分類: cs.CV, cs.AI, cs.LG, cs.RO
原文アブストラクト
Camera-radar (CR) fusion is a practical sensing configuration for autonomous driving, but existing models are typically trained with task-specific supervision, limiting reusable representation learning. We present CRISP, a spatiotemporal CR backbone pretrained through forecasting-based representation learning. Given historical multi-view images and radar sweeps, CRISP learns a unified bird's-eye-view (BEV) representation by predicting future LiDAR point clouds. LiDAR is used only as privileged supervision during pretraining; the deployed model requires only camera and radar. To make forecasting-based pretraining effective for CR fusion, CRISP introduces an enhanced radar encoder, radar-enhanced temporal self-attention, and multimodal feature rendering with modality innovation gating. These components inject radar range and Doppler cues into BEV temporal propagation and allow BEV tokens to selectively incorporate camera and radar evidence. Experiments on nuScenes show that CRISP improves long-horizon point cloud forecasting and transfers effectively to downstream tasks, including 3D detection, tracking, online mapping, motion forecasting, future occupancy prediction, and planning, suggesting that predictive CR pretraining is a promising path toward scalable driving representations under practical sensor configurations. The project website is https://umfieldrobotics.github.io/CRISP.
関連論文
- ラベルなし運転シナリオ複雑度検出のための統合埋め込み予測アーキテクチャ自動運転/自己教師あり学習
- VisionPAD: 自動運転のための視覚中心の事前学習パラダイム自動運転/自己教師あり学習
