ロボット学習のための言語・行動・状態の対照的事前学習
Contrastive Language, Action, and State Pre-training for Robot Learning
言語・行動・状態を共有埋め込み空間で結びつけるCLASPを提案し、ゼロショット検索やキャプション生成、言語条件付き強化学習の事前学習で有効性を示した。
著者: Krishan Rana, Andrew Melnik, Niko Sünderhauf
分類: cs.RO, cs.AI
原文アブストラクト
In this paper, we introduce a method for unifying language, action, and state information in a shared embedding space to facilitate a range of downstream tasks in robot learning. Our method, Contrastive Language, Action, and State Pre-training (CLASP), extends the CLIP formulation by incorporating distributional learning, capturing the inherent complexities and one-to-many relationships in behaviour-text alignment. By employing distributional outputs for both text and behaviour encoders, our model effectively associates diverse textual commands with a single behaviour and vice-versa. We demonstrate the utility of our method for the following downstream tasks: zero-shot text-behaviour retrieval, captioning unseen robot behaviours, and learning a behaviour prior for language-conditioned reinforcement learning. Our distributional encoders exhibit superior retrieval and captioning performance on unseen datasets, and the ability to generate meaningful exploratory behaviours from textual commands, capturing the intricate relationships between language, action, and state. This work represents an initial step towards developing a unified pre-trained model for robotics, with the potential to generalise to a broad range of downstream tasks.
