大規模シーンの自己教師あり新規2D視点合成と効率的なマルチスケールボクセルカービング
Self-supervised novel 2D view synthesis of large-scale scenes with efficient multi-scale voxel carving
UAVで撮影した大規模実環境シーンに対し、ノイズに頑健なマルチスケールボクセルカービングと自己教師あり学習を組み合わせ、高解像度の新規視点画像を生成する手法を提案した。
著者: Alexandra Budisteanu, Dragos Costea, Alina Marcu, Marius Leordeanu
分類: cs.CV, cs.LG, cs.RO
原文アブストラクト
The task of generating novel views of real scenes is increasingly important nowadays when AI models become able to create realistic new worlds. In many practical applications, it is important for novel view synthesis methods to stay grounded in the physical world as much as possible, while also being able to imagine it from previously unseen views. While most current methods are developed and tested in virtual environments with small scenes and no errors in pose and depth information, we push the boundaries to the real-world domain of large scales in the new context of UAVs. Our algorithmic contributions are two folds. First, we manage to stay anchored in the real 3D world, by introducing an efficient multi-scale voxel carving method, which is able to accommodate significant noises in pose, depth, and illumination variations, while being able to reconstruct the view of the world from drastically different poses at test time. Second, our final high-resolution output is efficiently self-trained on data automatically generated by the voxel carving module, which gives it the flexibility to adapt efficiently to any scene. We demonstrated the effectiveness of our method on highly complex and large-scale scenes in real environments while outperforming the current state-of-the-art. Our code is publicly available: https://github.com/onorabil/MSVC.
