Olaf-World: 動画世界モデリングのための潜在行動の方向づけ
Olaf-World: Orienting Latent Actions for Video World Modeling
ラベルなし動画から行動制御可能な世界モデルを学習するため、潜在行動を映像特徴の時間差分に整合させるSeqΔ-REPAを提案し、ゼロショット行動転移とデータ効率を改善した。
詳しい要約
1. どんなもの?
2. 先行研究と比べてどこがすごい?
3. 技術・手法の肝は?
4. どうやって有効だと検証した?
5. 議論はある?
6. 次に読むべき論文は?
※ AIが要旨から生成した要約です。正確性は原文をご確認ください。
著者: Yuxin Jiang, Yuchao Gu, Ivor W. Tsang, Mike Zheng Shou
分類: cs.CV, cs.AI, cs.LG
原文アブストラクト
Scaling action-controllable world models is limited by the scarcity of action labels. While latent action learning promises to extract control interfaces from unlabeled video, learned latents often fail to transfer across contexts: they entangle scene-specific cues and lack a shared coordinate system. This occurs because standard objectives operate only within each clip, providing no mechanism to align action semantics across contexts. Our key insight is that although actions are unobserved, their semantic effects are observable and can serve as a shared reference. We introduce Seq$Δ$-REPA, a sequence-level control-effect alignment objective that anchors integrated latent action to temporal feature differences from a frozen, self-supervised video encoder. Building on this, we present Olaf-World, a pipeline that pretrains action-conditioned video world models from large-scale passive video. Extensive experiments demonstrate that our method learns a more structured latent action space, leading to stronger zero-shot action transfer and more data-efficient adaptation to new control interfaces than state-of-the-art baselines.
