OSN: 単眼動画からの動的3Dシーンの無限表現
OSN: Infinite Representations of Dynamic 3D Scenes from Monocular Videos
単眼RGB動画から、入力に合致する全ての可能な動的3Dシーン表現を学習するフレームワークOSNを提案。物体スケールネットワークと共同最適化により、各動的物体の正確なスケール範囲を学習し、多様な3D配置を生成可能にした。
著者: Ziyang Song, Jinxi Li, Bo Yang
分類: cs.CV, cs.GR, cs.LG, cs.RO
原文アブストラクト
It has long been challenging to recover the underlying dynamic 3D scene representations from a monocular RGB video. Existing works formulate this problem into finding a single most plausible solution by adding various constraints such as depth priors and strong geometry constraints, ignoring the fact that there could be infinitely many 3D scene representations corresponding to a single dynamic video. In this paper, we aim to learn all plausible 3D scene configurations that match the input video, instead of just inferring a specific one. To achieve this ambitious goal, we introduce a new framework, called OSN. The key to our approach is a simple yet innovative object scale network together with a joint optimization module to learn an accurate scale range for every dynamic 3D object. This allows us to sample as many faithful 3D scene configurations as possible. Extensive experiments show that our method surpasses all baselines and achieves superior accuracy in dynamic novel view synthesis on multiple synthetic and real-world datasets. Most notably, our method demonstrates a clear advantage in learning fine-grained 3D scene geometry. Our code and data are available at https://github.com/vLAR-group/OSN
