日本フィジカルAI新聞

世界のフィジカルAIを、日本語で。

VLAarXiv:2407.12366

NavGPT-2:大規模視覚言語モデルのナビゲーション推論能力を引き出す

NavGPT-2: Unleashing Navigational Reasoning Capability for Large Vision-Language Models

シェア:XThreadsFacebookLINEはてブBluesky

凍結した大規模言語モデルに視覚情報を整合させ、ナビゲーション方策ネットワークと組み合わせることで、視覚言語ナビゲーションタスクにおける推論能力と行動予測性能を向上させた研究。

著者: Gengze Zhou, Yicong Hong, Zun Wang, Xin Eric Wang, Qi Wu

分類: cs.CV, cs.AI, cs.CL, cs.RO

原文アブストラクト

Capitalizing on the remarkable advancements in Large Language Models (LLMs), there is a burgeoning initiative to harness LLMs for instruction following robotic navigation. Such a trend underscores the potential of LLMs to generalize navigational reasoning and diverse language understanding. However, a significant discrepancy in agent performance is observed when integrating LLMs in the Vision-and-Language navigation (VLN) tasks compared to previous downstream specialist models. Furthermore, the inherent capacity of language to interpret and facilitate communication in agent interactions is often underutilized in these integrations. In this work, we strive to bridge the divide between VLN-specialized models and LLM-based navigation paradigms, while maintaining the interpretative prowess of LLMs in generating linguistic navigational reasoning. By aligning visual content in a frozen LLM, we encompass visual observation comprehension for LLMs and exploit a way to incorporate LLMs and navigation policy networks for effective action predictions and navigational reasoning. We demonstrate the data efficiency of the proposed methods and eliminate the gap between LM-based agents and state-of-the-art VLN specialists.

関連論文

  1. 1Boston Dynamics、車輪付きAtlasの研究試作機を紹介研究・技術
  2. 2newmoとKDDIが資本業務提携 2028年に堺市で自動運転タクシー商用化目指す提携・買収
  3. 3GXOとApptronik、Apolloの物流現場試験を拡大 最大1,000台の展開を構想量産・展開
  4. 4NVIDIA、CoRLでGR00T 2とIsaac Lab 3.0を発表新製品・発表
  5. 5UC BerkeleyとNVIDIAの研究者、AIエージェントでロボット制御コードを仮想環境で改善研究・技術

PR・アフィリエイト広告

MINISFORUM(ミニPC)

MINISFORUM(ミニPC)の公式広告バナーMINISFORUM公式ストアを見る ↗
PR本紙発行元 EmplifAI