視覚言語モデルによる間取り図マップの解析
Vision Language Models Can Parse Floor Plan Maps
視覚言語モデル(VLM)に間取り図を与え、複雑な屋内ナビゲーションのタスク計画を生成させることで、地図解析能力を評価した研究。
著者: David DeFazio, Hrudayangam Mehta, Meng Wang, Ping Yang, Jeremy Blackburn, Shiqi Zhang
分類: cs.RO, cs.AI
原文アブストラクト
Vision language models (VLMs) can simultaneously reason about images and texts to tackle many tasks, from visual question answering to image captioning. This paper focuses on map parsing, a novel task that is unexplored within the VLM context and particularly useful to mobile robots. Map parsing requires understanding not only the labels but also the geometric configurations of a map, i.e., what areas are like and how they are connected. To evaluate the performance of VLMs on map parsing, we prompt VLMs with floor plan maps to generate task plans for complex indoor navigation. Our results demonstrate the remarkable capability of VLMs in map parsing, with a success rate of 0.96 in tasks requiring a sequence of nine navigation actions, e.g., approaching and going through doors. Other than intuitive observations, e.g., VLMs do better in smaller maps and simpler navigation tasks, there was a very interesting observation that its performance drops in large open areas. We provide practical suggestions to address such challenges as validated by our experimental results. Webpage: https://sites.google.com/view/vlm-floorplan/
