N0-VTLA：潜在触覚トークンによる視覚・触覚・言語・行動モデルのスケーリング
$N_0$-VTLA: Scaling Vision-Tactile-Language-Action Model with Latent Tactile Tokens
触覚認識と触覚フィードバック制御を備えた視覚・触覚・言語・行動基盤モデルを提案し、大規模触覚データでの事前学習とオフライン強化学習によるポリシー改善を実現した。
詳しい要約
1. どんなもの？
2. 先行研究と比べてどこがすごい？
3. 技術・手法の肝は？
4. どうやって有効だと検証した？
5. 議論はある？
6. 次に読むべき論文は？
※ AIが要旨から生成した要約です。正確性は原文をご確認ください。
著者: NeoteAI Team, Fudan TEAI Team
分類: cs.RO
原文アブストラクト
We present $N_0$-VTLA, a vision-tactile-language-action (VTLA) foundation model capable of (1) fine-grained contact-rich manipulation with tactile perception and tactile-feedback control, and (2) offline policy improvement from stored deployment data. Building on current vision-based backbones, we propose a training recipe for tactile integration consisting of visuo-tactile pre-training, staged tactile-pathway integration, and advantage-conditioned offline policy improvement. During pre-training, the policy learns broad contact priors from NeoData, our large-scale visuo-tactile robot dataset; to our knowledge, $N_0$-VTLA is the first VTLA model pretrained on tactile data at scale. During post-training, we augment the policy with a predictive tactile pathway that distills the contact patterns learned at scale into the fine motion adjustments required by downstream tactile-centric manipulation. For offline policy improvement, we introduce ALTER, an advantage-conditioned offline reinforcement learning method that converts relative progress and trajectory-event comparisons into binary advantage labels for policy training on a fixed deployment corpus, further improving task-specific learning on contact-rich skills such as deformable object manipulation. Across contact-rich benchmarks, $N_0$-VTLA outperforms strong baselines by wide margins: it wins all nine real-robot NeoReal tasks and reaches 63.8% mean success on a twenty-task simulation suite, against 44.0% for the strongest baseline. $N_0$-VTLA policies trained with ALTER reach 75-95% success on three long-horizon real-robot tasks. These results lay a foundation for versatile tactile-driven manipulation policies.
関連論文
- ViTaR: 基盤VLA操作のための視触覚残差適応VLA/触覚/操作
- TACO: 触覚を考慮した世界モデルによるスケーラブルなVLAポストトレーニングの自己修正VLA/触覚/操作
- τ: 未来の視覚的監督から学習する触覚拡張型視覚言語行動モデルVLA/触覚/操作
- AT-VLA: 適応的触覚注入による視覚言語行動モデルのフィードバック反応向上VLA/触覚/操作
- TacVLA: 接触認識型触覚融合による堅牢な視覚言語行動操作VLA/触覚/操作
