少数例から物体を接地・地図化する自然言語ロボット指示追従
Few-shot Object Grounding and Mapping for Natural Language Robot Instruction Following
拡張現実データから学習した少数例ベースの物体接地手法と、物体の位置と指示された用途を符号化する学習地図表現を提案し、実機クアッドコプターの連続制御に統合して未知物体にも対応できることを示した。
著者: Valts Blukis, Ross A. Knepper, Yoav Artzi
分類: cs.RO, cs.AI, cs.CL, cs.CV, cs.LG
原文アブストラクト
We study the problem of learning a robot policy to follow natural language instructions that can be easily extended to reason about new objects. We introduce a few-shot language-conditioned object grounding method trained from augmented reality data that uses exemplars to identify objects and align them to their mentions in instructions. We present a learned map representation that encodes object locations and their instructed use, and construct it from our few-shot grounding output. We integrate this mapping approach into an instruction-following policy, thereby allowing it to reason about previously unseen objects at test-time by simply adding exemplars. We evaluate on the task of learning to map raw observations and instructions to continuous control of a physical quadcopter. Our approach significantly outperforms the prior state of the art in the presence of new objects, even when the prior approach observes all objects during training.
