自律走行のためのニューラルレンダリングによる都市シーン再構成
Neural Rendering based Urban Scene Reconstruction for Autonomous Driving
LiDARの正確な疎な深度とカメラの密な深度を組み合わせ、ニューラル暗黙的表面と放射輝度場を統合したフレームワークで都市の3Dシーンを高精度に再構成する手法を提案。
著者: Shihao Shen, Louis Kerofsky, Varun Ravi Kumar, Senthil Yogamani
分類: cs.CV, cs.RO
原文アブストラクト
Dense 3D reconstruction has many applications in automated driving including automated annotation validation, multimodal data augmentation, providing ground truth annotations for systems lacking LiDAR, as well as enhancing auto-labeling accuracy. LiDAR provides highly accurate but sparse depth, whereas camera images enable estimation of dense depth but noisy particularly at long ranges. In this paper, we harness the strengths of both sensors and propose a multimodal 3D scene reconstruction using a framework combining neural implicit surfaces and radiance fields. In particular, our method estimates dense and accurate 3D structures and creates an implicit map representation based on signed distance fields, which can be further rendered into RGB images, and depth maps. A mesh can be extracted from the learned signed distance field and culled based on occlusion. Dynamic objects are efficiently filtered on the fly during sampling using 3D object detection models. We demonstrate qualitative and quantitative results on challenging automotive scenes.
