文脈的観察者接地：視覚言語モデルにおける状況的空間推論の評価
Contextual Observer Grounding: Evaluating Situated Spatial Reasoning in Vision-Language Models
ロボットなどの身体化タスクで、話者の視点に基づく空間関係の理解を評価するベンチマークPOVBenchを構築し、最新の視覚言語モデルが文脈から観察者の視点を推論して対象を特定できるかを検証した。
詳しい要約
1. どんなもの？
2. 先行研究と比べてどこがすごい？
3. 技術・手法の肝は？
4. どうやって有効だと検証した？
5. 議論はある？
6. 次に読むべき論文は？
※ AIが要旨から生成した要約です。正確性は原文をご確認ください。
著者: Mimo Shirasaka, Haochen Zhang, Yonatan Bisk
分類: cs.CV, cs.CL, cs.LG, cs.RO
原文アブストラクト
Reasoning over language instructions in embodied tasks such as robotics often requires understanding spatial relations from a speaker's situated perspective. Humans infer such perspectives from shared environmental knowledge, activity context, and commonsense. Recent vision-language models (VLMs) appear capable of spatial reasoning, but their ability to infer a speaker's viewpoint from contextual cues and interpret situated spatial relations from that viewpoint remains unclear. We call this capability contextual observer grounding. To study this capability, we construct the Point-of-View Benchmark (POVBench), a dataset of 3D scenes and queries that disentangles Inferred, Stated, and Given forms of observer grounding in natural embodied communication. Given multi-view observations and a natural-language sentence, models must localize unseen or underspecified targets from situated spatial and contextual cues. Across state-of-the-art VLMs, localizing targets from directional language remains challenging, even when observer grounding is made explicit. We find that explicit breakdowns of observer-relative spatial reasoning improve target localization. Our project page is available at https://mimo-owl.github.io/POVBench/.
関連論文
- 空間思考の連鎖：視覚言語モデルのためのモダリティ非依存の空間接地VLA/空間推論
