R4: 4次元時空間における視覚言語モデルのための検索拡張推論
R4: Retrieval-Augmented Reasoning for Vision-Language Models in 4D Spatio-Temporal Space
視覚言語モデルに4次元時空間の構造化された生涯記憶を持たせ、訓練なしで検索拡張推論を行うフレームワークR4を提案。物体レベルの意味情報を空間と時間に紐づけて蓄積し、クエリを意味・空間・時間のキーに分解して検索することで、動的環境でのエピソード的・協調的推論を可能にする。
詳しい要約
1. どんなもの?
2. 先行研究と比べてどこがすごい?
3. 技術・手法の肝は?
4. どうやって有効だと検証した?
5. 議論はある?
6. 次に読むべき論文は?
※ AIが要旨から生成した要約です。正確性は原文をご確認ください。
著者: Tin Stribor Sohn, Maximilian Dillitzer, Jason J. Corso, Eric Sax
分類: cs.CV, cs.RO
原文アブストラクト
Humans perceive and reason about their surroundings in four dimensions by building persistent, structured internal representations that encode semantic meaning, spatial layout, and temporal dynamics. These multimodal memories enable them to recall past events, infer unobserved states, and integrate new information into context-dependent reasoning. Inspired by this capability, we introduce R4, a training-free framework for retrieval-augmented reasoning in 4D spatio-temporal space that equips vision-language models (VLMs) with structured, lifelong memory. R4 continuously constructs a 4D knowledge database by anchoring object-level semantic descriptions in metric space and time, yielding a persistent world model that can be shared across agents. At inference, natural language queries are decomposed into semantic, spatial, and temporal keys to retrieve relevant observations, which are integrated into the VLM's reasoning. Unlike classical retrieval-augmented generation methods, retrieval in R4 operates directly in 4D space, enabling episodic and collaborative reasoning without training. Experiments on embodied question answering and navigation benchmarks demonstrate that R4 substantially improves retrieval and reasoning over spatio-temporal information compared to baselines, advancing a new paradigm for embodied 4D reasoning in dynamic environments.
