事前学習済み視覚言語モデルのVQAによるロボットの二値状態認識
Binary State Recognition by Robots using Visual Question Answering of Pre-Trained Vision-Language Model
大規模データで事前学習済みの視覚言語モデルに視覚的質問応答(VQA)を適用し、再学習なしでドアの開閉や把持の成否などの二値状態を言語で認識する手法を検討・評価した。
著者: Kento Kawaharazuka, Yoshiki Obinata, Naoaki Kanazawa, Kei Okada, Masayuki Inaba
分類: cs.RO
原文アブストラクト
Recognition of the current state is indispensable for the operation of a robot. There are various states to be recognized, such as whether an elevator door is open or closed, whether an object has been grasped correctly, and whether the TV is turned on or off. Until now, these states have been recognized by programmatically describing the state of a point cloud or raw image, by annotating and learning images, by using special sensors, etc. In contrast to these methods, we apply Visual Question Answering (VQA) from a Pre-Trained Vision-Language Model (PTVLM) trained on a large-scale dataset, to such binary state recognition. This idea allows us to intuitively describe state recognition in language without any re-training, thereby improving the recognition ability of robots in a simple and general way. We summarize various techniques in questioning methods and image processing, and clarify their properties through experiments.
