Semi-SMD: 自動運転向け周囲カメラによる半教師ありメトリック深度推定
Semi-SMD: Semi-Supervised Metric Depth Estimation via Surrounding Cameras for Autonomous Driving
自動運転車の周囲カメラ映像とカメラパラメータから、時空間・意味情報を統合してメトリック深度を半教師ありで推定するフレームワークを提案し、DDADとnuScenesで最先端性能を達成した。
著者: Yusen Xie, Zhengmin Huang, Shaojie Shen, Jun Ma
分類: cs.RO, cs.CV
原文アブストラクト
In this paper, we introduce Semi-SMD, a novel metric depth estimation framework tailored for surrounding cameras equipment in autonomous driving. In this work, the input data consists of adjacent surrounding frames and camera parameters. We propose a unified spatial-temporal-semantic fusion module to construct the visual fused features. Cross-attention components for surrounding cameras and adjacent frames are utilized to focus on metric scale information refinement and temporal feature matching. Building on this, we propose a pose estimation framework using surrounding cameras, their corresponding estimated depths, and extrinsic parameters, which effectively address the scale ambiguity in multi-camera setups. Moreover, semantic world model and monocular depth estimation world model are integrated to supervised the depth estimation, which improve the quality of depth estimation. We evaluate our algorithm on DDAD and nuScenes datasets, and the results demonstrate that our method achieves state-of-the-art performance in terms of surrounding camera based depth estimation quality. The source code will be available on https://github.com/xieyuser/Semi-SMD.
