Aug3D：大規模屋外データセットを拡張し汎化可能な新規視点合成を実現
Aug3D: Augmenting large scale outdoor datasets for Generalizable Novel View Synthesis
大規模屋外データセットUrbanScene3Dでフィードフォワード型NVSモデルを訓練し、SfM再構成を活用したAug3D拡張で視点重複不足を補い新規視点予測性能を向上させた研究。
著者: Aditya Rauniyar, Omar Alama, Silong Yong, Katia Sycara, Sebastian Scherer
分類: cs.CV, cs.AI, cs.RO
原文アブストラクト
Recent photorealistic Novel View Synthesis (NVS) advances have increasingly gained attention. However, these approaches remain constrained to small indoor scenes. While optimization-based NVS models have attempted to address this, generalizable feed-forward methods, offering significant advantages, remain underexplored. In this work, we train PixelNeRF, a feed-forward NVS model, on the large-scale UrbanScene3D dataset. We propose four training strategies to cluster and train on this dataset, highlighting that performance is hindered by limited view overlap. To address this, we introduce Aug3D, an augmentation technique that leverages reconstructed scenes using traditional Structure-from-Motion (SfM). Aug3D generates well-conditioned novel views through grid and semantic sampling to enhance feed-forward NVS model learning. Our experiments reveal that reducing the number of views per cluster from 20 to 10 improves PSNR by 10%, but the performance remains suboptimal. Aug3D further addresses this by combining the newly generated novel views with the original dataset, demonstrating its effectiveness in improving the model's ability to predict novel views.
