視覚から触覚・音響へのクロスモーダル転移学習による潜在物体特性認識
Latent Object Characteristics Recognition with Visual to Haptic-Audio Cross-modal Transfer Learning
視覚で学習した潜在表現を触覚・音響・モータデータへ転移学習し、ロボットが把持中の物体の形状・位置・姿勢を間接センサ情報から認識できるようにした研究。
著者: Namiko Saito, Joao Moura, Hiroki Uchida, Sethu Vijayakumar
分類: cs.RO, cs.CV, cs.LG
原文アブストラクト
Recognising the characteristics of objects while a robot handles them is crucial for adjusting motions that ensure stable and efficient interactions with containers. Ahead of realising stable and efficient robot motions for handling/transferring the containers, this work aims to recognise the latent unobservable object characteristics. While vision is commonly used for object recognition by robots, it is ineffective for detecting hidden objects. However, recognising objects indirectly using other sensors is a challenging task. To address this challenge, we propose a cross-modal transfer learning approach from vision to haptic-audio. We initially train the model with vision, directly observing the target object. Subsequently, we transfer the latent space learned from vision to a second module, trained only with haptic-audio and motor data. This transfer learning framework facilitates the representation of object characteristics using indirect sensor data, thereby improving recognition accuracy. For evaluating the recognition accuracy of our proposed learning framework we selected shape, position, and orientation as the object characteristics. Finally, we demonstrate online recognition of both trained and untrained objects using the humanoid robot Nextage Open.
