MSSF: 自律走行における3D物体検出のためのマルチステージサンプリングを用いた4Dレーダーとカメラの融合フレームワーク
MSSF: A 4D Radar and Camera Fusion Framework With Multi-Stage Sampling for 3D Object Detection in Autonomous Driving
4Dミリ波レーダーとカメラを融合し、マルチステージサンプリングと意味誘導ヘッドで3D物体検出の精度を高める手法を提案した論文。
著者: Hongsi Liu, Jun Liu, Guangfeng Jiang, Xin Jin
分類: cs.CV, cs.RO
原文アブストラクト
As one of the automotive sensors that have emerged in recent years, 4D millimeter-wave radar has a higher resolution than conventional 3D radar and provides precise elevation measurements. But its point clouds are still sparse and noisy, making it challenging to meet the requirements of autonomous driving. Camera, as another commonly used sensor, can capture rich semantic information. As a result, the fusion of 4D radar and camera can provide an affordable and robust perception solution for autonomous driving systems. However, previous radar-camera fusion methods have not yet been thoroughly investigated, resulting in a large performance gap compared to LiDAR-based methods. Specifically, they ignore the feature-blurring problem and do not deeply interact with image semantic information. To this end, we present a simple but effective multi-stage sampling fusion (MSSF) network based on 4D radar and camera. On the one hand, we design a fusion block that can deeply interact point cloud features with image features, and can be applied to commonly used single-modal backbones in a plug-and-play manner. The fusion block encompasses two types, namely, simple feature fusion (SFF) and multiscale deformable feature fusion (MSDFF). The SFF is easy to implement, while the MSDFF has stronger fusion abilities. On the other hand, we propose a semantic-guided head to perform foreground-background segmentation on voxels with voxel feature re-weighting, further alleviating the problem of feature blurring. Extensive experiments on the View-of-Delft (VoD) and TJ4DRadset datasets demonstrate the effectiveness of our MSSF. Notably, compared to state-of-the-art methods, MSSF achieves a 7.0% and 4.0% improvement in 3D mean average precision on the VoD and TJ4DRadSet datasets, respectively. It even surpasses classical LiDAR-based methods on the VoD dataset.
関連論文
- SDCM: 車載ネットワークにおけるレーダー・視覚3次元物体検出のための模擬稠密化と補償モデリング融合センサーフュージョン
- CaRLi-V: カメラ・レーダー・LiDAR融合による点単位3D速度推定センサーフュージョン
- カメラによる道路レイアウト抽出を活用したT字路でのmmWaveレーダーによる見通し外歩行者位置推定センサーフュージョン
- 不確かさ駆動型レーダー慣性融合による瞬時3次元自己速度推定センサーフュージョン
- 任意のLiDARをガイドとする拡散事前分布による教師なしレーダー点群強調センサーフュージョン
- 画像と4Dレーダースペクトルの深度認識融合による3D物体検出センサーフュージョン
