オンライン3Dガウススプラッティングによる明示的メモリでクラス非依存動画セグメンテーションを改善
Explicit Memory through Online 3D Gaussian Splatting Improves Class-Agnostic Video Segmentation
動画中の物体セグメントをオンライン3Dガウススプラッティングで明示的に記憶し、FastSAMやSAM2の予測精度と一貫性を向上させる手法を提案。
詳しい要約
1. どんなもの?
2. 先行研究と比べてどこがすごい?
3. 技術・手法の肝は?
4. どうやって有効だと検証した?
5. 議論はある?
6. 次に読むべき論文は?
※ AIが要旨から生成した要約です。正確性は原文をご確認ください。
著者: Anthony Opipari, Aravindhan K Krishnan, Shreekant Gayaka, Min Sun, Cheng-Hao Kuo, Arnie Sen, Odest Chadwicke Jenkins
分類: cs.RO
原文アブストラクト
Remembering where object segments were predicted in the past is useful for improving the accuracy and consistency of class-agnostic video segmentation algorithms. Existing video segmentation algorithms typically use either no object-level memory (e.g. FastSAM) or they use implicit memories in the form of recurrent neural network features (e.g. SAM2). In this paper, we augment both types of segmentation models using an explicit 3D memory and show that the resulting models have more accurate and consistent predictions. For this, we develop an online 3D Gaussian Splatting (3DGS) technique to store predicted object-level segments generated throughout the duration of a video. Based on this 3DGS representation, a set of fusion techniques are developed, named FastSAM-Splat and SAM2-Splat, that use the explicit 3DGS memory to improve their respective foundation models' predictions. Ablation experiments are used to validate the proposed techniques' design and hyperparameter settings. Results from both real-world and simulated benchmarking experiments show that models which use explicit 3D memories result in more accurate and consistent predictions than those which use no memory or only implicit neural network memories. Project Page: https://topipari.com/projects/FastSAM-Splat/
関連論文
- 遮蔽をマスキングとして扱う動画アモーダルセグメンテーション動画セグメンテーション
- DynEoMT: オンラインセグメンテーションクエリから物体の動的性を学習動画セグメンテーション
- 効率的なワンショット動画物体セグメンテーションの再考動画セグメンテーション
