一貫したシーンセグメントを用いた意味的密再構成
Semantic Dense Reconstruction with Consistent Scene Segments
RGB-D列から高精度な3D密再構成と意味セグメンテーションを同時に行う手法を提案し、2Dの一貫したセグメントと3Dモデルを活用した意味投影ブロックで特徴を融合する。
著者: Yingcai Wan, Yanyan Li, Yingxuan You, Cheng Guo, Lijin Fang, Federico Tombari
分類: cs.CV, cs.RO
原文アブストラクト
In this paper, a method for dense semantic 3D scene reconstruction from an RGB-D sequence is proposed to solve high-level scene understanding tasks. First, each RGB-D pair is consistently segmented into 2D semantic maps based on a camera tracking backbone that propagates objects' labels with high probabilities from full scans to corresponding ones of partial views. Then a dense 3D mesh model of an unknown environment is incrementally generated from the input RGB-D sequence. Benefiting from 2D consistent semantic segments and the 3D model, a novel semantic projection block (SP-Block) is proposed to extract deep feature volumes from 2D segments of different views. Moreover, the semantic volumes are fused into deep volumes from a point cloud encoder to make the final semantic segmentation. Extensive experimental evaluations on public datasets show that our system achieves accurate 3D dense reconstruction and state-of-the-art semantic prediction performances simultaneously.
