動作焦点の潜在アクションによる人間のエゴビデオからのクロスエンボディVLA訓練
Motion-Focused Latent Action Enables Cross-Embodiment VLA Training from Human EgoVideos
ラベルなしの人間のエゴビデオから動作の潜在アクションを抽出し、ロボットのVLAモデルを訓練する手法を提案。シミュレーションと実環境で、わずか50トラジェクトリの適応で大規模データセットで訓練した最先端モデルと同等の性能を達成。
詳しい要約
1. どんなもの？
2. 先行研究と比べてどこがすごい？
3. 技術・手法の肝は？
4. どうやって有効だと検証した？
5. 議論はある？
6. 次に読むべき論文は？
※ AIが要旨から生成した要約です。正確性は原文をご確認ください。
著者: Runze Xu, Yiluo Zhang, Jian Wang, Yu Wang, Jincheng Yu
分類: cs.CV, cs.RO
原文アブストラクト
Training generalist Vision-Language-Action(VLA) models typically requires massive, diverse robotic datasets with high-fidelity action annotations. While egocentric human manipulation videos are abundant and capture significant environmental diversity, the absence of action labels makes them difficult to use in conventional training paradigms. To address this, we propose a latent-action-based framework designed to extract general action priors from unlabeled human videos. The architecture features a Hybrid Disentangled VQ-VAE that decouples motion dynamics from environmental backgrounds through physical masks, enabling the construction of a cross-embodiment action codebook. By pre-training on human videos with the codebook, the VLM backbone learns deep representations of action intent. For adaptation to specific embodiments, we introduce an intent-perception decoupling strategy where the VLM predicts the action intent while a separate frozen visual encoder provides state-specific features to the action expert, thereby reducing action hallucinations. Results in simulation and real-world environments show that our method, pre-trained exclusively on unlabeled human videos, performs competitively with state-of-the-art VLA models trained on massive annotated datasets, requiring only 50 trajectories for downstream adaptation.
