日本フィジカルAI新聞

世界のフィジカルAIを、日本語で。

視覚言語ナビゲーションarXiv:2002.10638

事前学習による汎用視覚言語ナビゲーションエージェントの学習

Towards Learning a Generic Agent for Vision-and-Language Navigation via Pre-training

シェア:XThreadsFacebookLINEはてブBluesky

視覚言語ナビゲーション(VLN)タスク向けに初の事前学習・微調整パラダイムを提案し、大量の画像・テキスト・行動の三つ組で自己教師あり学習を行うことで汎用的な表現を獲得し、複数タスクで最先端性能を達成した。

著者: Weituo Hao, Chunyuan Li, Xiujun Li, Lawrence Carin, Jianfeng Gao

分類: cs.CV, cs.CL, cs.LG, cs.RO

原文アブストラクト

Learning to navigate in a visual environment following natural-language instructions is a challenging task, because the multimodal inputs to the agent are highly variable, and the training data on a new task is often limited. In this paper, we present the first pre-training and fine-tuning paradigm for vision-and-language navigation (VLN) tasks. By training on a large amount of image-text-action triplets in a self-supervised learning manner, the pre-trained model provides generic representations of visual environments and language instructions. It can be easily used as a drop-in for existing VLN frameworks, leading to the proposed agent called Prevalent. It learns more effectively in new tasks and generalizes better in a previously unseen environment. The performance is validated on three VLN tasks. On the Room-to-Room benchmark, our model improves the state-of-the-art from 47% to 51% on success rate weighted by path length. Further, the learned representation is transferable to other VLN tasks. On two recent tasks, vision-and-dialog navigation and "Help, Anna!" the proposed Prevalent leads to significant improvement over existing methods, achieving a new state of the art.

関連論文

  1. 1Boston Dynamics、車輪付きAtlasの研究試作機を紹介研究・技術
  2. 2newmoとKDDIが資本業務提携 2028年に堺市で自動運転タクシー商用化目指す提携・買収
  3. 3GXOとApptronik、Apolloの物流現場試験を拡大 最大1,000台の展開を構想量産・展開
  4. 4NVIDIA、CoRLでGR00T 2とIsaac Lab 3.0を発表新製品・発表
  5. 5UC BerkeleyとNVIDIAの研究者、AIエージェントでロボット制御コードを仮想環境で改善研究・技術

PR・アフィリエイト広告

OZ GAMING(GPU搭載BTOゲーミングPC)

OZ GAMING(GPU搭載BTOゲーミングPC)の公式広告バナーOZ GAMINGで完成品PCを見る ↗
PR本紙発行元 EmplifAI