VisionPAD: 自動運転のための視覚中心の事前学習パラダイム
VisionPAD: A Vision-Centric Pre-training Paradigm for Autonomous Driving
画像のみを教師信号に3Dガウシアンスプラッティングで多視点表現を再構成し、ボクセル速度推定と多フレーム測光整合性で自動運転向け視覚モデルを自己教師あり事前学習する手法。
著者: Haiming Zhang, Wending Zhou, Yiyao Zhu, Xu Yan, Jiantao Gao, Dongfeng Bai, Yingjie Cai, Bingbing Liu, Shuguang Cui, Zhen Li
分類: cs.CV, cs.LG, cs.RO
原文アブストラクト
This paper introduces VisionPAD, a novel self-supervised pre-training paradigm designed for vision-centric algorithms in autonomous driving. In contrast to previous approaches that employ neural rendering with explicit depth supervision, VisionPAD utilizes more efficient 3D Gaussian Splatting to reconstruct multi-view representations using only images as supervision. Specifically, we introduce a self-supervised method for voxel velocity estimation. By warping voxels to adjacent frames and supervising the rendered outputs, the model effectively learns motion cues in the sequential data. Furthermore, we adopt a multi-frame photometric consistency approach to enhance geometric perception. It projects adjacent frames to the current frame based on rendered depths and relative poses, boosting the 3D geometric representation through pure image supervision. Extensive experiments on autonomous driving datasets demonstrate that VisionPAD significantly improves performance in 3D object detection, occupancy prediction and map segmentation, surpassing state-of-the-art pre-training strategies by a considerable margin.
関連論文
- CRISP: 予測型ワールドモデル事前学習による運転用時空間カメラ・レーダーバックボーン自動運転/自己教師あり学習
- ラベルなし運転シナリオ複雑度検出のための統合埋め込み予測アーキテクチャ自動運転/自己教師あり学習
