RGB-D動画から変形可能物体モデルを学習する粒子-グリッドニューラルダイナミクス
Particle-Grid Neural Dynamics for Learning Deformable Object Models from RGB-D Videos
粒子と空間グリッドを組み合わせたハイブリッド表現で、ロボットと物体の相互作用を撮影したRGB-D動画から、ロープや布などの変形可能物体のダイナミクスを学習し、モデルベース計画に応用する手法を提案。
詳しい要約
1. どんなもの？
2. 先行研究と比べてどこがすごい？
3. 技術・手法の肝は？
4. どうやって有効だと検証した？
5. 議論はある？
6. 次に読むべき論文は？
※ AIが要旨から生成した要約です。正確性は原文をご確認ください。
著者: Kaifeng Zhang, Baoyu Li, Kris Hauser, Yunzhu Li
分類: cs.RO, cs.CV, cs.LG
原文アブストラクト
Modeling the dynamics of deformable objects is challenging due to their diverse physical properties and the difficulty of estimating states from limited visual information. We address these challenges with a neural dynamics framework that combines object particles and spatial grids in a hybrid representation. Our particle-grid model captures global shape and motion information while predicting dense particle movements, enabling the modeling of objects with varied shapes and materials. Particles represent object shapes, while the spatial grid discretizes the 3D space to ensure spatial continuity and enhance learning efficiency. Coupled with Gaussian Splattings for visual rendering, our framework achieves a fully learning-based digital twin of deformable objects and generates 3D action-conditioned videos. Through experiments, we demonstrate that our model learns the dynamics of diverse objects -- such as ropes, cloths, stuffed animals, and paper bags -- from sparse-view RGB-D recordings of robot-object interactions, while also generalizing at the category level to unseen instances. Our approach outperforms state-of-the-art learning-based and physics-based simulators, particularly in scenarios with limited camera views. Furthermore, we showcase the utility of our learned models in model-based planning, enabling goal-conditioned object manipulation across a range of tasks. The project page is available at https://kywind.github.io/pgnd .
関連論文
- ビデオからモデルへ：変形可能な線状物体の自動モデリング変形可能物体モデリング
- 変形可能な線状物体のリアルタイムモデリングのための微分可能な離散弾性ロッド変形可能物体モデリング
