SceneFactory: インクリメンタルなシーン構築のためのワークフロー中心の統合フレームワーク
SceneFactory: A Workflow-centric and Unified Framework for Incremental Scene Modeling
多視点深度推定やLiDAR補完、RGB-D/RGB-L/単眼/深度のみの再構成、SLAMなど多様な応用を統一的に扱うワークフロー中心のフレームワークを提案。追跡・屈曲・深度推定・シーン再構成の4ブロックで構成され、姿勢・キャリブレーション不要の多視点深度推定モデルU²-MVDを導入。
著者: Yijun Yuan, Michael Bleier, Andreas Nüchter
分類: cs.CV, cs.RO
原文アブストラクト
We present SceneFactory, a workflow-centric and unified framework for incremental scene modeling, that conveniently supports a wide range of applications, such as (unposed and/or uncalibrated) multi-view depth estimation, LiDAR completion, (dense) RGB-D/RGB-L/Mono/Depth-only reconstruction and SLAM. The workflow-centric design uses multiple blocks as the basis for constructing different production lines. The supported applications, i.e., productions avoid redundancy in their designs. Thus, the focus is placed on each block itself for independent expansion. To support all input combinations, our implementation consists of four building blocks that form SceneFactory: (1) tracking, (2) flexion, (3) depth estimation, and (4) scene reconstruction. The tracking block is based on Mono SLAM and is extended to support RGB-D and RGB-LiDAR (RGB-L) inputs. Flexion is used to convert the depth image (untrackable) into a trackable image. For general-purpose depth estimation, we propose an unposed \& uncalibrated multi-view depth estimation model (U$^2$-MVD) to estimate dense geometry. U$^2$-MVD exploits dense bundle adjustment to solve for poses, intrinsics, and inverse depth. A semantic-aware ScaleCov step is then introduced to complete the multi-view depth. Relying on U$^2$-MVD, SceneFactory both supports user-friendly 3D creation (with just images) and bridges the applications of Dense RGB-D and Dense Mono. For high-quality surface and color reconstruction, we propose Dual-purpose Multi-resolutional Neural Points (DM-NPs) for the first surface accessible Surface Color Field design, where we introduce Improved Point Rasterization (IPR) for point cloud based surface query. ...
