FreeGave: ガウス速度による動画からの3D物理学習
FreeGave: 3D Physics Learning from Dynamic Videos by Gaussian Velocity
多視点動画から物体の事前情報なしに3Dシーンの物理運動を学習する手法を提案。ガウス速度場を推定し、未来フレーム予測と動きのセグメンテーションで高性能を示す。
詳しい要約
1. どんなもの?
2. 先行研究と比べてどこがすごい?
3. 技術・手法の肝は?
4. どうやって有効だと検証した?
5. 議論はある?
6. 次に読むべき論文は?
※ AIが要旨から生成した要約です。正確性は原文をご確認ください。
著者: Jinxi Li, Ziyang Song, Siyuan Zhou, Bo Yang
分類: cs.CV, cs.AI, cs.CE, cs.LG, cs.RO
原文アブストラクト
In this paper, we aim to model 3D scene geometry, appearance, and the underlying physics purely from multi-view videos. By applying various governing PDEs as PINN losses or incorporating physics simulation into neural networks, existing works often fail to learn complex physical motions at boundaries or require object priors such as masks or types. In this paper, we propose FreeGave to learn the physics of complex dynamic 3D scenes without needing any object priors. The key to our approach is to introduce a physics code followed by a carefully designed divergence-free module for estimating a per-Gaussian velocity field, without relying on the inefficient PINN losses. Extensive experiments on three public datasets and a newly collected challenging real-world dataset demonstrate the superior performance of our method for future frame extrapolation and motion segmentation. Most notably, our investigation into the learned physics codes reveals that they truly learn meaningful 3D physical motion patterns in the absence of any human labels in training.
