遺伝的アルゴリズムで最適化したVQAによるロボット状態認識
VQA-based Robotic State Recognition Optimized with Genetic Algorithm
事前学習済み視覚言語モデルに視覚質問応答を適用し、遺伝的アルゴリズムで質問の組み合わせを探索することで、透明ドアや水の状態など従来困難だったロボットの状態認識を実現した。
著者: Kento Kawaharazuka, Yoshiki Obinata, Naoaki Kanazawa, Kei Okada, Masayuki Inaba
分類: cs.RO
原文アブストラクト
State recognition of objects and environment in robots has been conducted in various ways. In most cases, this is executed by processing point clouds, learning images with annotations, and using specialized sensors. In contrast, in this study, we propose a state recognition method that applies Visual Question Answering (VQA) in a Pre-Trained Vision-Language Model (PTVLM) trained from a large-scale dataset. By using VQA, it is possible to intuitively describe robotic state recognition in the spoken language. On the other hand, there are various possible ways to ask about the same event, and the performance of state recognition differs depending on the question. Therefore, in order to improve the performance of state recognition using VQA, we search for an appropriate combination of questions using a genetic algorithm. We show that our system can recognize not only the open/closed of a refrigerator door and the on/off of a display, but also the open/closed of a transparent door and the state of water, which have been difficult to recognize.
