LLM-RG: 大規模言語モデルを用いた屋外シーンにおける参照表現のグラウンディング
LLM-RG: Referential Grounding in Outdoor Scenarios using Large Language Models
屋外運転シーンで自然言語の参照表現(例:「右側の黒い車」)が指す物体を特定するため、視覚言語モデルで属性を抽出し、大規模言語モデルで推論するハイブリッド手法を提案。Talk2Carベンチマークで大幅な精度向上を達成。
詳しい要約
1. どんなもの?
2. 先行研究と比べてどこがすごい?
3. 技術・手法の肝は?
4. どうやって有効だと検証した?
5. 議論はある?
6. 次に読むべき論文は?
※ AIが要旨から生成した要約です。正確性は原文をご確認ください。
著者: Pranav Saxena, Avigyan Bhattacharya, Ji Zhang, Wenshan Wang
分類: cs.CV, cs.AI, cs.RO
原文アブストラクト
Referential grounding in outdoor driving scenes is challenging due to large scene variability, many visually similar objects, and dynamic elements that complicate resolving natural-language references (e.g., "the black car on the right"). We propose LLM-RG, a hybrid pipeline that combines off-the-shelf vision-language models for fine-grained attribute extraction with large language models for symbolic reasoning. LLM-RG processes an image and a free-form referring expression by using an LLM to extract relevant object types and attributes, detecting candidate regions, generating rich visual descriptors with a VLM, and then combining these descriptors with spatial metadata into natural-language prompts that are input to an LLM for chain-of-thought reasoning to identify the referent's bounding box. Evaluated on the Talk2Car benchmark, LLM-RG yields substantial gains over both LLM and VLM-based baselines. Additionally, our ablations show that adding 3D spatial cues further improves grounding. Our results demonstrate the complementary strengths of VLMs and LLMs, applied in a zero-shot manner, for robust outdoor referential grounding.
