モーメントベース敵対的学習による身体性言語理解
Moment-based Adversarial Training for Embodied Language Comprehension
家庭内タスク指示をサブゴールに分解して実行する視覚言語タスクに対し、指示・サブゴール・状態表現の埋め込み空間に2種類のモーメントを用いた敵対的学習を導入し、ALFREDベンチマークで全指標においてベースラインを上回った。
著者: Shintaro Ishikawa, Komei Sugiura
分類: cs.RO, cs.CL, cs.CV
原文アブストラクト
In this paper, we focus on a vision-and-language task in which a robot is instructed to execute household tasks. Given an instruction such as "Rinse off a mug and place it in the coffee maker," the robot is required to locate the mug, wash it, and put it in the coffee maker. This is challenging because the robot needs to break down the instruction sentences into subgoals and execute them in the correct order. On the ALFRED benchmark, the performance of state-of-the-art methods is still far lower than that of humans. This is partially because existing methods sometimes fail to infer subgoals that are not explicitly specified in the instruction sentences. We propose Moment-based Adversarial Training (MAT), which uses two types of moments for perturbation updates in adversarial training. We introduce MAT to the embedding spaces of the instruction, subgoals, and state representations to handle their varieties. We validated our method on the ALFRED benchmark, and the results demonstrated that our method outperformed the baseline method for all the metrics on the benchmark.
