USegScene: 意味的ガイダンスと結合ネットワークによる深度・オプティカルフロー・自己運動の教師なし学習
USegScene: Unsupervised Learning of Depth, Optical Flow and Ego-Motion with Semantic Guidance and Coupled Networks
ステレオ画像から深度・オプティカルフロー・自己運動を教師なしで推定するフレームワークを提案し、意味情報を活用した正則化や遮蔽処理、共有エンコーダによるマルチタスク学習で高精度を実現した。
著者: Johan Vertens, Wolfram Burgard
分類: cs.CV, cs.RO
原文アブストラクト
In this paper we propose USegScene, a framework for semantically guided unsupervised learning of depth, optical flow and ego-motion estimation for stereo camera images using convolutional neural networks. Our framework leverages semantic information for improved regularization of depth and optical flow maps, multimodal fusion and occlusion filling considering dynamic rigid object motions as independent SE(3) transformations. Furthermore, complementary to pure photo-metric matching, we propose matching of semantic features, pixel-wise classes and object instance borders between the consecutive images. In contrast to previous methods, we propose a network architecture that jointly predicts all outputs using shared encoders and allows passing information across the task-domains, e.g., the prediction of optical flow can benefit from the prediction of the depth. Furthermore, we explicitly learn the depth and optical flow occlusion maps inside the network, which are leveraged in order to improve the predictions in therespective regions. We present results on the popular KITTI dataset and show that our approach outperforms other methods by a large margin.
