GR-2: ウェブ規模の知識を持つ生成型ビデオ・言語・行動モデルによるロボットマニピュレーション
GR-2: A Generative Video-Language-Action Model with Web-Scale Knowledge for Robot Manipulation
3800万本のインターネット動画で事前学習した大規模モデルをロボット軌道で微調整し、100以上のタスクで平均97.7%の成功率と未知環境への高い汎化性能を実現した。
著者: Chi-Lam Cheang, Guangzeng Chen, Ya Jing, Tao Kong, Hang Li, Yifeng Li, Yuxiao Liu, Hongtao Wu, Jiafeng Xu, Yichu Yang, Hanbo Zhang, Minzhao Zhu
分類: cs.RO, cs.CV, cs.LG
原文アブストラクト
We present GR-2, a state-of-the-art generalist robot agent for versatile and generalizable robot manipulation. GR-2 is first pre-trained on a vast number of Internet videos to capture the dynamics of the world. This large-scale pre-training, involving 38 million video clips and over 50 billion tokens, equips GR-2 with the ability to generalize across a wide range of robotic tasks and environments during subsequent policy learning. Following this, GR-2 is fine-tuned for both video generation and action prediction using robot trajectories. It exhibits impressive multi-task learning capabilities, achieving an average success rate of 97.7% across more than 100 tasks. Moreover, GR-2 demonstrates exceptional generalization to new, previously unseen scenarios, including novel backgrounds, environments, objects, and tasks. Notably, GR-2 scales effectively with model size, underscoring its potential for continued growth and application. Project page: \url{https://gr2-manipulation.github.io}.
