NeRFを活用した不確実性を考慮した視覚的自己位置推定
Leveraging Neural Radiance Fields for Uncertainty-Aware Visual Localization
NeRFで生成した学習データの不確実性を評価し、情報量の多い視点を選別することで、シーン座標回帰による視覚的自己位置推定を高効率化する手法を提案。
著者: Le Chen, Weirong Chen, Rui Wang, Marc Pollefeys
分類: cs.CV, cs.RO
原文アブストラクト
As a promising fashion for visual localization, scene coordinate regression (SCR) has seen tremendous progress in the past decade. Most recent methods usually adopt neural networks to learn the mapping from image pixels to 3D scene coordinates, which requires a vast amount of annotated training data. We propose to leverage Neural Radiance Fields (NeRF) to generate training samples for SCR. Despite NeRF's efficiency in rendering, many of the rendered data are polluted by artifacts or only contain minimal information gain, which can hinder the regression accuracy or bring unnecessary computational costs with redundant data. These challenges are addressed in three folds in this paper: (1) A NeRF is designed to separately predict uncertainties for the rendered color and depth images, which reveal data reliability at the pixel level. (2) SCR is formulated as deep evidential learning with epistemic uncertainty, which is used to evaluate information gain and scene coordinate quality. (3) Based on the three arts of uncertainties, a novel view selection policy is formed that significantly improves data efficiency. Experiments on public datasets demonstrate that our method could select the samples that bring the most information gain and promote the performance with the highest efficiency.
