事前学習済み表現による汎化可能な模倣学習
Generalizable Imitation Learning Through Pre-Trained Representations
自己教師ありViTのパッチ埋め込みをクラスタリングして意味的なキーポイントを形成し、外観変化や物体種別に汎化する模倣学習手法DVKを提案した。
著者: Wei-Di Chang, Francois Hogan, Scott Fujimoto, David Meger, Gregory Dudek
分類: cs.RO, cs.AI
原文アブストラクト
In this paper, we leverage self-supervised vision transformer models and their emergent semantic abilities to improve the generalization abilities of imitation learning policies. We introduce DVK, an imitation learning algorithm that leverages rich pre-trained Visual Transformer patch-level embeddings to obtain better generalization when learning through demonstrations. Our learner sees the world by clustering appearance features into groups associated with semantic concepts, forming stable keypoints that generalize across a wide range of appearance variations and object types. We demonstrate how this representation enables generalized behaviour by evaluating imitation learning across a diverse dataset of object manipulation tasks. To facilitate further study of generalization in Imitation Learning, all of our code for the method and evaluation, as well as the dataset, is made available.
