視覚言語モデルによる視触覚ゼロショット物体認識
Visuo-Tactile Zero-Shot Object Recognition with Vision-Language Model
触覚データを視覚言語モデル(VLM)に組み込み、物体名の注釈だけで触覚をテキスト化してゼロショットで物体を認識する手法を提案した。
著者: Shiori Ueda, Atsushi Hashimoto, Masashi Hamaya, Kazutoshi Tanaka, Hideo Saito
分類: cs.RO
原文アブストラクト
Tactile perception is vital, especially when distinguishing visually similar objects. We propose an approach to incorporate tactile data into a Vision-Language Model (VLM) for visuo-tactile zero-shot object recognition. Our approach leverages the zero-shot capability of VLMs to infer tactile properties from the names of tactilely similar objects. The proposed method translates tactile data into a textual description solely by annotating object names for each tactile sequence during training, making it adaptable to various contexts with low training costs. The proposed method was evaluated on the FoodReplica and Cube datasets, demonstrating its effectiveness in recognizing objects that are difficult to distinguish by vision alone.
