日本フィジカルAI新聞

世界のフィジカルAIを、日本語で。

VLAarXiv:2502.01218

汎用身体性エージェントのための視覚言語事前学習における順序性と連続性の証明可能な学習

Provable Ordering and Continuity in Vision-Language Pretraining for Generalizable Embodied Agents

シェア:XThreadsFacebookLINEはてブBluesky

人間の行動動画から視覚言語表現を事前学習する際に、目標到達に基づく時間対比学習ではなく、フレーム間の意味的差異とブラウン橋制約を用いて順序性と連続性を学習するAcTOLを提案し、模倣学習実験で操縦タスクの性能と指示スタイルへの頑健性を向上させた。

著者: Zhizhen Zhang, Lei Zhu, Zhen Fang, Zi Huang, Yadan Luo

分類: cs.RO, cs.AI, cs.CV, cs.LG

原文アブストラクト

Pre-training vision-language representations on human action videos has emerged as a promising approach to reduce reliance on large-scale expert demonstrations for training embodied agents. However, prior methods often employ time contrastive learning based on goal-reaching heuristics, progressively aligning language instructions from the initial to the final frame. This overemphasis on future frames can result in erroneous vision-language associations, as actions may terminate early or include irrelevant moments in the end. To address this issue, we propose Action Temporal Coherence Learning (AcTOL) to learn ordered and continuous vision-language representations without rigid goal-based constraint. AcTOL treats a video as a continuous trajectory where it (1) contrasts semantic differences between frames to reflect their natural ordering, and (2) imposes a local Brownian bridge constraint to ensure smooth transitions across intermediate frames. Extensive imitation learning experiments on both simulated and real robots show that the pretrained features significantly enhance downstream manipulation tasks with high robustness to different linguistic styles of instructions, offering a viable pathway toward generalized embodied agents.

関連論文

  1. 1Boston Dynamics、車輪付きAtlasの研究試作機を紹介研究・技術
  2. 2newmoとKDDIが資本業務提携 2028年に堺市で自動運転タクシー商用化目指す提携・買収
  3. 3GXOとApptronik、Apolloの物流現場試験を拡大 最大1,000台の展開を構想量産・展開
  4. 4NVIDIA、CoRLでGR00T 2とIsaac Lab 3.0を発表新製品・発表
  5. 5UC BerkeleyとNVIDIAの研究者、AIエージェントでロボット制御コードを仮想環境で改善研究・技術

PR・アフィリエイト広告

FRONTIER(GPU搭載BTOパソコン)

FRONTIER(GPU搭載BTOパソコン)の公式広告バナーFRONTIERでGPU搭載PCを見る ↗
PR本紙発行元 EmplifAI