視覚言語モデルによるエンドツーエンドナビゲーション：空間推論を質問応答に変換
End-to-End Navigation with Vision Language Models: Transforming Spatial Reasoning into Question-Answering
視覚言語モデル（VLM）をファインチューニングなしで直接ナビゲーション方策として使い、知覚・計画・制御を分離せずに行動選択を行うエンドツーエンド手法を提案。
著者: Dylan Goetting, Himanshu Gaurav Singh, Antonio Loquercio
分類: cs.RO, cs.CL, cs.CV
原文アブストラクト
We present VLMnav, an embodied framework to transform a Vision-Language Model (VLM) into an end-to-end navigation policy. In contrast to prior work, we do not rely on a separation between perception, planning, and control; instead, we use a VLM to directly select actions in one step. Surprisingly, we find that a VLM can be used as an end-to-end policy zero-shot, i.e., without any fine-tuning or exposure to navigation data. This makes our approach open-ended and generalizable to any downstream navigation task. We run an extensive study to evaluate the performance of our approach in comparison to baseline prompting methods. In addition, we perform a design analysis to understand the most impactful design decisions. Visual examples and code for our project can be found at https://jirl-upenn.github.io/VLMnav/
