単眼動画からのインスタンス別深度・運動学習
Instance-wise Depth and Motion Learning from Monocular Videos
単眼動画から複数の動的物体の6自由度運動・自己運動・深度を教師なしで同時学習するフレームワークを提案し、KITTIで従来手法を上回った。
著者: Seokju Lee, Sunghoon Im, Stephen Lin, In So Kweon
分類: cs.CV, cs.LG, cs.RO
原文アブストラクト
We present an end-to-end joint training framework that explicitly models 6-DoF motion of multiple dynamic objects, ego-motion and depth in a monocular camera setup without supervision. Our technical contributions are three-fold. First, we propose a differentiable forward rigid projection module that plays a key role in our instance-wise depth and motion learning. Second, we design an instance-wise photometric and geometric consistency loss that effectively decomposes background and moving object regions. Lastly, we introduce a new auto-annotation scheme to produce video instance segmentation maps that will be utilized as input to our training pipeline. These proposed elements are validated in a detailed ablation study. Through extensive experiments conducted on the KITTI dataset, our framework is shown to outperform the state-of-the-art depth and motion estimation methods. Our code and dataset will be available at https://github.com/SeokjuLee/Insta-DM.
