DeepLiDARFlow:単眼カメラとスパースLiDARを用いたシーンフローの深層学習アーキテクチャ
DeepLiDARFlow: A Deep Learning Architecture For Scene Flow Estimation Using Monocular Camera and Sparse LiDAR
単眼RGB画像とスパースLiDARを複数スケールで融合し、高密度なシーンフローを推定する深層学習手法を提案。反射や影など画像のみ・LiDARのみが苦手な領域で高精度を実現。
著者: Rishav, Ramy Battrawy, René Schuster, Oliver Wasenmüller, Didier Stricker
分類: cs.CV, cs.LG, cs.RO
原文アブストラクト
Scene flow is the dense 3D reconstruction of motion and geometry of a scene. Most state-of-the-art methods use a pair of stereo images as input for full scene reconstruction. These methods depend a lot on the quality of the RGB images and perform poorly in regions with reflective objects, shadows, ill-conditioned light environment and so on. LiDAR measurements are much less sensitive to the aforementioned conditions but LiDAR features are in general unsuitable for matching tasks due to their sparse nature. Hence, using both LiDAR and RGB can potentially overcome the individual disadvantages of each sensor by mutual improvement and yield robust features which can improve the matching process. In this paper, we present DeepLiDARFlow, a novel deep learning architecture which fuses high level RGB and LiDAR features at multiple scales in a monocular setup to predict dense scene flow. Its performance is much better in the critical regions where image-only and LiDAR-only methods are inaccurate. We verify our DeepLiDARFlow using the established data sets KITTI and FlyingThings3D and we show strong robustness compared to several state-of-the-art methods which used other input modalities. The code of our paper is available at https://github.com/dfki-av/DeepLiDARFlow.
