幾何誘導コントラスト学習によるスペクトル別・マルチスペクトル深度推定の橋渡し
Bridging Spectral-wise and Multi-spectral Depth Estimation via Geometry-guided Contrastive Learning
マルチスペクトル画像から深度を推定するため、幾何情報を手がかりにコントラスト学習でスペクトル間の埋め込み空間を整列し、選択的に特徴を統合するモジュールを学習する手法を提案。
著者: Ukcheol Shin, Kyunghyun Lee, Jean Oh
分類: cs.CV, cs.AI, cs.RO
原文アブストラクト
Deploying depth estimation networks in the real world requires high-level robustness against various adverse conditions to ensure safe and reliable autonomy. For this purpose, many autonomous vehicles employ multi-modal sensor systems, including an RGB camera, NIR camera, thermal camera, LiDAR, or Radar. They mainly adopt two strategies to use multiple sensors: modality-wise and multi-modal fused inference. The former method is flexible but memory-inefficient, unreliable, and vulnerable. Multi-modal fusion can provide high-level reliability, yet it needs a specialized architecture. In this paper, we propose an effective solution, named align-and-fuse strategy, for the depth estimation from multi-spectral images. In the align stage, we align embedding spaces between multiple spectrum bands to learn shareable representation across multi-spectral images by minimizing contrastive loss of global and spatially aligned local features with geometry cue. After that, in the fuse stage, we train an attachable feature fusion module that can selectively aggregate the multi-spectral features for reliable and robust prediction results. Based on the proposed method, a single-depth network can achieve both spectral-invariant and multi-spectral fused depth estimation while preserving reliability, memory efficiency, and flexibility.
