Cog-GA: 大規模言語モデルによる連続環境での視覚言語ナビゲーション生成エージェント
Cog-GA: A Large Language Models-based Generative Agent for Vision-Language Navigation in Continuous Environments
大規模言語モデルを基盤に、認知マップ構築・ウェイポイント予測・二チャネル場面記述・反省機構を組み合わせ、連続3D環境での視覚言語ナビゲーションを人間らしく効率的に行う生成エージェントを提案した。
著者: Zhiyuan Li, Yanfeng Lu, Yao Mu, Hong Qiao
分類: cs.AI, cs.RO
原文アブストラクト
Vision Language Navigation in Continuous Environments (VLN-CE) represents a frontier in embodied AI, demanding agents to navigate freely in unbounded 3D spaces solely guided by natural language instructions. This task introduces distinct challenges in multimodal comprehension, spatial reasoning, and decision-making. To address these challenges, we introduce Cog-GA, a generative agent founded on large language models (LLMs) tailored for VLN-CE tasks. Cog-GA employs a dual-pronged strategy to emulate human-like cognitive processes. Firstly, it constructs a cognitive map, integrating temporal, spatial, and semantic elements, thereby facilitating the development of spatial memory within LLMs. Secondly, Cog-GA employs a predictive mechanism for waypoints, strategically optimizing the exploration trajectory to maximize navigational efficiency. Each waypoint is accompanied by a dual-channel scene description, categorizing environmental cues into 'what' and 'where' streams as the brain. This segregation enhances the agent's attentional focus, enabling it to discern pertinent spatial information for navigation. A reflective mechanism complements these strategies by capturing feedback from prior navigation experiences, facilitating continual learning and adaptive replanning. Extensive evaluations conducted on VLN-CE benchmarks validate Cog-GA's state-of-the-art performance and ability to simulate human-like navigation behaviors. This research significantly contributes to the development of strategic and interpretable VLN-CE agents.
