3D人間ポーズで基盤視覚モデルを接地させたロバスト行動認識
Grounding Foundational Vision Models with 3D Human Poses for Robust Action Recognition
V-JEPA 2の予測的世界ダイナミクスとCoMotionの遮蔽に強い3D人間ポーズを融合し、物理空間に接地した行動認識モデルを提案。高遮蔽シーンで既存手法を上回る。
詳しい要約
1. どんなもの?
2. 先行研究と比べてどこがすごい?
3. 技術・手法の肝は?
4. どうやって有効だと検証した?
5. 議論はある?
6. 次に読むべき論文は?
※ AIが要旨から生成した要約です。正確性は原文をご確認ください。
著者: Nicholas Babey, Tiffany Gu, Yiheng Li, Cristian Meo, Kevin Zhu
分類: cs.CV, cs.AI, cs.LG, cs.RO
原文アブストラクト
For embodied agents to effectively understand and interact within the world around them, they require a nuanced comprehension of human actions grounded in physical space. Current action recognition models, often relying on RGB video, learn superficial correlations between patterns and action labels, so they struggle to capture underlying physical interaction dynamics and human poses in complex scenes. We propose a model architecture that grounds action recognition in physical space by fusing two powerful, complementary representations: V-JEPA 2's contextual, predictive world dynamics and CoMotion's explicit, occlusion-tolerant human pose data. Our model is validated on both the InHARD and UCF-19-Y-OCC benchmarks for general action recognition and high-occlusion action recognition, respectively. Our model outperforms three other baselines, especially within complex, occlusive scenes. Our findings emphasize a need for action recognition to be supported by spatial understanding instead of statistical pattern recognition.
