合成データで実世界のカメラ位置推定をスケールさせるCrossLoc
CrossLoc: Scalable Aerial Localization Assisted by Multimodal Synthetic Data
実世界と仮想世界を滑らかにつなぐ合成データ生成ツールTOPO-DataGenと大規模sim2realベンチマークを提案し、自己教師あり学習で合成データを活用する位置推定手法CrossLocを開発した。
著者: Qi Yan, Jianhao Zheng, Simon Reding, Shanci Li, Iordan Doytchinov
分類: cs.CV, cs.AI, cs.RO
原文アブストラクト
We present a visual localization system that learns to estimate camera poses in the real world with the help of synthetic data. Despite significant progress in recent years, most learning-based approaches to visual localization target at a single domain and require a dense database of geo-tagged images to function well. To mitigate the data scarcity issue and improve the scalability of the neural localization models, we introduce TOPO-DataGen, a versatile synthetic data generation tool that traverses smoothly between the real and virtual world, hinged on the geographic camera viewpoint. New large-scale sim-to-real benchmark datasets are proposed to showcase and evaluate the utility of the said synthetic data. Our experiments reveal that synthetic data generically enhances the neural network performance on real data. Furthermore, we introduce CrossLoc, a cross-modal visual representation learning approach to pose estimation that makes full use of the scene coordinate ground truth via self-supervision. Without any extra data, CrossLoc significantly outperforms the state-of-the-art methods and achieves substantially higher real-data sample efficiency. Our code and datasets are all available at https://crossloc.github.io/.
