身体性を活かした視覚認識：遮蔽物越しの物体理解のための能動的移動学習
Embodied Visual Recognition
3D環境内でエージェントが戦略的に移動しながら遮蔽された物体を分類・位置推定・セグメンテーションするEmbodied Visual Recognitionタスクを提案し、Embodied Mask R-CNNで最短経路とは異なる戦略的移動を学習できることを示した。
著者: Jianwei Yang, Zhile Ren, Mingze Xu, Xinlei Chen, David Crandall, Devi Parikh, Dhruv Batra
分類: cs.CV, cs.AI, cs.LG, cs.RO
原文アブストラクト
Passive visual systems typically fail to recognize objects in the amodal setting where they are heavily occluded. In contrast, humans and other embodied agents have the ability to move in the environment, and actively control the viewing angle to better understand object shapes and semantics. In this work, we introduce the task of Embodied Visual Recognition (EVR): An agent is instantiated in a 3D environment close to an occluded target object, and is free to move in the environment to perform object classification, amodal object localization, and amodal object segmentation. To address this, we develop a new model called Embodied Mask R-CNN, for agents to learn to move strategically to improve their visual recognition abilities. We conduct experiments using the House3D environment. Experimental results show that: 1) agents with embodiment (movement) achieve better visual recognition performance than passive ones; 2) in order to improve visual recognition abilities, agents can learn strategical moving paths that are different from shortest paths.
