ソフト常識制約を用いた探索によるゼロショット物体ナビゲーション
ESC: Exploration with Soft Commonsense Constraints for Zero-shot Object Navigation
事前学習済みの視覚言語モデルと常識言語モデルを活用し、ナビゲーション経験なしで未知環境の物体へ到達するゼロショット物体ナビゲーション手法ESCを提案。
著者: Kaiwen Zhou, Kaizhi Zheng, Connor Pryor, Yilin Shen, Hongxia Jin, Lise Getoor, Xin Eric Wang
分類: cs.AI, cs.CV, cs.LG, cs.RO
原文アブストラクト
The ability to accurately locate and navigate to a specific object is a crucial capability for embodied agents that operate in the real world and interact with objects to complete tasks. Such object navigation tasks usually require large-scale training in visual environments with labeled objects, which generalizes poorly to novel objects in unknown environments. In this work, we present a novel zero-shot object navigation method, Exploration with Soft Commonsense constraints (ESC), that transfers commonsense knowledge in pre-trained models to open-world object navigation without any navigation experience nor any other training on the visual environments. First, ESC leverages a pre-trained vision and language model for open-world prompt-based grounding and a pre-trained commonsense language model for room and object reasoning. Then ESC converts commonsense knowledge into navigation actions by modeling it as soft logic predicates for efficient exploration. Extensive experiments on MP3D, HM3D, and RoboTHOR benchmarks show that our ESC method improves significantly over baselines, and achieves new state-of-the-art results for zero-shot object navigation (e.g., 288% relative Success Rate improvement than CoW on MP3D).
関連論文
- 動く前に探索方針を決める:拡散モデル生成の探索参照を用いた半静的環境でのゼロショット物体ナビゲーション物体ナビゲーション
- HTCE:同相写像に着想を得た物体ナビゲーションのためのシーン横断経験の正規化と再利用物体ナビゲーション
- SOR-Nav: 探索継続か領域移動か?文脈ゲート付き探索と領域間移動による物体ナビゲーション物体ナビゲーション
- NaviScale: 物体ナビゲーション向け大規模セマンティックマップデータセットの生成物体ナビゲーション
- ユーザー中心物体ナビゲーション:個人の習慣を統合したパーソナライズド物体探索ベンチマーク物体ナビゲーション
- Hydra-Nav: 適応的二重プロセス推論による物体ナビゲーション物体ナビゲーション
