物体中心3Dモーションフィールドによる人間動画からのロボット学習
Object-centric 3D Motion Field for Robot Learning from Human Videos
人間の動画から物体中心の3Dモーションフィールドを抽出し、ロボットのゼロショット制御を可能にするフレームワークを提案。ノイズの多い深度から頑健に動きを推定し、多様なタスクで高い成功率を達成。
詳しい要約
1. どんなもの？
2. 先行研究と比べてどこがすごい？
3. 技術・手法の肝は？
4. どうやって有効だと検証した？
5. 議論はある？
6. 次に読むべき論文は？
※ AIが要旨から生成した要約です。正確性は原文をご確認ください。
著者: Zhao-Heng Yin, Sherry Yang, Pieter Abbeel
分類: cs.RO, cs.AI, cs.CV, cs.LG, cs.SY, eess.SY
原文アブストラクト
Learning robot control policies from human videos is a promising direction for scaling up robot learning. However, how to extract action knowledge (or action representations) from videos for policy learning remains a key challenge. Existing action representations such as video frames, pixelflow, and pointcloud flow have inherent limitations such as modeling complexity or loss of information. In this paper, we propose to use object-centric 3D motion field to represent actions for robot learning from human videos, and present a novel framework for extracting this representation from videos for zero-shot control. We introduce two novel components in its implementation. First, a novel training pipeline for training a ''denoising'' 3D motion field estimator to extract fine object 3D motions from human videos with noisy depth robustly. Second, a dense object-centric 3D motion field prediction architecture that favors both cross-embodiment transfer and policy generalization to background. We evaluate the system in real world setups. Experiments show that our method reduces 3D motion estimation error by over 50% compared to the latest method, achieve 55% average success rate in diverse tasks where prior approaches fail~($\lesssim 10$\%), and can even acquire fine-grained manipulation skills like insertion.
