MLOD:堅牢な特徴融合に基づくマルチビュー3D物体検出
MLOD: A multi-view 3D object detection based on robust feature fusion method
RGB画像とLiDAR点群を入力とし、BEV上の3D提案を画像・BEV両方の特徴マップに投影して融合する2段階3D物体検出器MLODを提案。深度マスクと各センサチャネル個別の検出ヘッドにより特徴抽出器の劣化を防ぎ、KITTIで最先端性能を達成した。
著者: Jian Deng, Krzysztof Czarnecki
分類: cs.CV, cs.LG, cs.RO
原文アブストラクト
This paper presents Multi-view Labelling Object Detector (MLOD). The detector takes an RGB image and a LIDAR point cloud as input and follows the two-stage object detection framework. A Region Proposal Network (RPN) generates 3D proposals in a Bird's Eye View (BEV) projection of the point cloud. The second stage projects the 3D proposal bounding boxes to the image and BEV feature maps and sends the corresponding map crops to a detection header for classification and bounding-box regression. Unlike other multi-view based methods, the cropped image features are not directly fed to the detection header, but masked by the depth information to filter out parts outside 3D bounding boxes. The fusion of image and BEV features is challenging, as they are derived from different perspectives. We introduce a novel detection header, which provides detection results not just from fusion layer, but also from each sensor channel. Hence the object detector can be trained on data labelled in different views to avoid the degeneration of feature extractors. MLOD achieves state-of-the-art performance on the KITTI 3D object detection benchmark. Most importantly, the evaluation shows that the new header architecture is effective in preventing image feature extractor degeneration.
