MAEA: 身体性AIのためのマルチモーダル帰属分析
MAEA: Multimodal Attribution for Embodied AI
ALFREDデータセットで訓練された方策に対し、視覚・言語・過去行動の各モダリティの寄与度を大域的に算出する枠組みMAEAを提案し、失敗事例の分類やバイアス分析に活用できることを示した。
著者: Vidhi Jain, Jayant Sravan Tamarapalli, Sahiti Yerramilli, Yonatan Bisk
分類: cs.LG, cs.AI, cs.CV, cs.RO
原文アブストラクト
Understanding multimodal perception for embodied AI is an open question because such inputs may contain highly complementary as well as redundant information for the task. A relevant direction for multimodal policies is understanding the global trends of each modality at the fusion layer. To this end, we disentangle the attributions for visual, language, and previous action inputs across different policies trained on the ALFRED dataset. Attribution analysis can be utilized to rank and group the failure scenarios, investigate modeling and dataset biases, and critically analyze multimodal EAI policies for robustness and user trust before deployment. We present MAEA, a framework to compute global attributions per modality of any differentiable policy. In addition, we show how attributions enable lower-level behavior analysis in EAI policies for language and visual attributions.
