内視鏡動画からの深度推定とカメラ軌跡復元
Endo-Depth-and-Motion: Reconstruction and Tracking in Endoscopic Videos using Depth Networks and Photometric Constraints
自己教師あり深度ネットワークで単眼内視鏡動画から擬似RGBDフレームを生成し、光度残差でカメラ姿勢を追跡して密な3D再構成を行うパイプラインを提案した論文。
著者: David Recasens, José Lamarca, José M. Fácil, J. M. M. Montiel, Javier Civera
分類: cs.CV, cs.LG, cs.RO
原文アブストラクト
Estimating a scene reconstruction and the camera motion from in-body videos is challenging due to several factors, e.g. the deformation of in-body cavities or the lack of texture. In this paper we present Endo-Depth-and-Motion, a pipeline that estimates the 6-degrees-of-freedom camera pose and dense 3D scene models from monocular endoscopic videos. Our approach leverages recent advances in self-supervised depth networks to generate pseudo-RGBD frames, then tracks the camera pose using photometric residuals and fuses the registered depth maps in a volumetric representation. We present an extensive experimental evaluation in the public dataset Hamlyn, showing high-quality results and comparisons against relevant baselines. We also release all models and code for future comparisons.
