身体性タスク完了のためのマルチモーダル文脈化プラン予測
Multimodal Contextualized Plan Prediction for Embodied Task Completion
身体性エージェントのタスク完了データセットTEAChにおいて、自然言語から高レベルなプラン表現を予測する手法を提案し、マルチモーダル文脈の活用とプラン予測・実行の相互依存性を示した。
著者: Mert İnan, Aishwarya Padmakumar, Spandana Gella, Patrick Lange, Dilek Hakkani-Tur
分類: cs.RO, cs.AI, cs.CL, cs.HC
原文アブストラクト
Task planning is an important component of traditional robotics systems enabling robots to compose fine grained skills to perform more complex tasks. Recent work building systems for translating natural language to executable actions for task completion in simulated embodied agents is focused on directly predicting low level action sequences that would be expected to be directly executable by a physical robot. In this work, we instead focus on predicting a higher level plan representation for one such embodied task completion dataset - TEACh, under the assumption that techniques for high-level plan prediction from natural language are expected to be more transferable to physical robot systems. We demonstrate that better plans can be predicted using multimodal context, and that plan prediction and plan execution modules are likely dependent on each other and hence it may not be ideal to fully decouple them. Further, we benchmark execution of oracle plans to quantify the scope for improvement in plan prediction models.
