時間的グラウンディンググラフによる視覚言語文脈を蓄積した言語理解
Temporal Grounding Graphs for Language Understanding with Accrued Visual-Linguistic Context
自然言語指示と過去の視覚観察・対話から得た事実知識を統合し、指示対象や将来行動を確率的に推定して計画を立てるロボット向け手法を提案。
著者: Rohan Paul, Andrei Barbu, Sue Felshin, Boris Katz, Nicholas Roy
分類: cs.RO
原文アブストラクト
A robot's ability to understand or ground natural language instructions is fundamentally tied to its knowledge about the surrounding world. We present an approach to grounding natural language utterances in the context of factual information gathered through natural-language interactions and past visual observations. A probabilistic model estimates, from a natural language utterance, the objects,relations, and actions that the utterance refers to, the objectives for future robotic actions it implies, and generates a plan to execute those actions while updating a state representation to include newly acquired knowledge from the visual-linguistic context. Grounding a command necessitates a representation for past observations and interactions; however, maintaining the full context consisting of all possible observed objects, attributes, spatial relations, actions, etc., over time is intractable. Instead, our model, Temporal Grounding Graphs, maintains a learned state representation for a belief over factual groundings, those derived from natural-language interactions, and lazily infers new groundings from visual observations using the context implied by the utterance. This work significantly expands the range of language that a robot can understand by incorporating factual knowledge and observations of its workspace in its inference about the meaning and grounding of natural-language utterances.
関連論文
- ロボット言語グラウンディングのサーベイ:記号と埋め込みのトレードオフ言語グラウンディング
- どっち?物体間の文脈と複数視点を活用した言語グラウンディング言語グラウンディング
- 言語グラウンディングにおける信頼:人間とロボットのチームに向けた新たなAI課題言語グラウンディング
- 領域別動的レイヤーアテンションによる層融合を用いた自動運転車のコマンドグラウンディング言語グラウンディング
- ジェスチャーと言語理解を組み合わせた屋内ロボットの認知共有型ナビゲーション言語グラウンディング
- 視覚知覚に基づく言語獲得のためのニューラル変分学習言語グラウンディング
