語り、夢見て、行動する：指示駆動型ロボット操作のためのビデオ世界モデル学習
Say, Dream, and Act: Learning Video World Models for Instruction-Driven Robot Manipulation
ビデオ生成モデルを蒸留して高速化し、生成された未来映像と実観測を組み合わせて行動モデルを訓練することで、指示に基づくロボット操作の精度と効率を向上させるフレームワークを提案。
詳しい要約
1. どんなもの？
2. 先行研究と比べてどこがすごい？
3. 技術・手法の肝は？
4. どうやって有効だと検証した？
5. 議論はある？
6. 次に読むべき論文は？
※ AIが要旨から生成した要約です。正確性は原文をご確認ください。
著者: Songen Gu, Yunuo Cai, Tianyu Wang, Simo Wu, Yanwei Fu
分類: cs.RO
原文アブストラクト
Robotic manipulation requires anticipating how the environment evolves in response to actions, yet most existing systems lack this predictive capability, often resulting in errors and inefficiency. While Vision-Language Models (VLMs) provide high-level guidance, they cannot explicitly forecast future states, and existing world models either predict only short horizons or produce spatially inconsistent frames. To address these challenges, we propose a framework for fast and predictive video-conditioned action. Our approach first selects and adapts a robust video generation model to ensure reliable future predictions, then applies adversarial distillation for fast, few-step video generation, and finally trains an action model that leverages both generated videos and real observations to correct spatial errors. Extensive experiments show that our method produces temporally coherent, spatially accurate video predictions that directly support precise manipulation, achieving significant improvements in embodiment consistency, spatial referring ability, and task completion over existing baselines. Codes & Models will be released.
