物体配置のための自然言語指示の微分可能な構文解析と視覚的グラウンディング
Differentiable Parsing and Visual Grounding of Natural Language Instructions for Object Placement
自然言語による物体配置指示を、構文解析と粒子ベースのグラフニューラルネットワークで物体ごとにグラウンディングし、曖昧さを考慮しつつエンドツーエンドで学習する手法を提案。
著者: Zirui Zhao, Wee Sun Lee, David Hsu
分類: cs.RO, cs.CV, cs.LG
原文アブストラクト
We present a new method, PARsing And visual GrOuNding (ParaGon), for grounding natural language in object placement tasks. Natural language generally describes objects and spatial relations with compositionality and ambiguity, two major obstacles to effective language grounding. For compositionality, ParaGon parses a language instruction into an object-centric graph representation to ground objects individually. For ambiguity, ParaGon uses a novel particle-based graph neural network to reason about object placements with uncertainty. Essentially, ParaGon integrates a parsing algorithm into a probabilistic, data-driven learning framework. It is fully differentiable and trained end-to-end from data for robustness against complex, ambiguous language input.
