動的シーンの4D表現を学習するニューラル放射フロー
Neural Radiance Flow for 4D View Synthesis and Video Processing
RGB画像から動的シーンの3D占有・放射・動きを暗黙的に学習し、多視点レンダリングや動画の超解像・ノイズ除去を可能にする手法を提案。
著者: Yilun Du, Yinan Zhang, Hong-Xing Yu, Joshua B. Tenenbaum, Jiajun Wu
分類: cs.CV, cs.LG, cs.RO
原文アブストラクト
We present a method, Neural Radiance Flow (NeRFlow),to learn a 4D spatial-temporal representation of a dynamic scene from a set of RGB images. Key to our approach is the use of a neural implicit representation that learns to capture the 3D occupancy, radiance, and dynamics of the scene. By enforcing consistency across different modalities, our representation enables multi-view rendering in diverse dynamic scenes, including water pouring, robotic interaction, and real images, outperforming state-of-the-art methods for spatial-temporal view synthesis. Our approach works even when inputs images are captured with only one camera. We further demonstrate that the learned representation can serve as an implicit scene prior, enabling video processing tasks such as image super-resolution and de-noising without any additional supervision.
