対照学習とデータ拡張による効率的な視覚ロボット制御の学習
Learning Visual Robotic Control Efficiently with Contrastive Pre-training and Data Augmentation
対照学習による事前学習とデータ拡張を組み合わせ、実機ロボットアームが10個のデモンストレーションから30分程度の訓練でスパース報酬の操作タスクをピクセル入力で学習できる手法CoDERを提案した。
著者: Albert Zhan, Ruihan Zhao, Lerrel Pinto, Pieter Abbeel, Michael Laskin
分類: cs.RO, cs.AI, cs.LG
原文アブストラクト
Recent advances in unsupervised representation learning significantly improved the sample efficiency of training Reinforcement Learning policies in simulated environments. However, similar gains have not yet been seen for real-robot reinforcement learning. In this work, we focus on enabling data-efficient real-robot learning from pixels. We present Contrastive Pre-training and Data Augmentation for Efficient Robotic Learning (CoDER), a method that utilizes data augmentation and unsupervised learning to achieve sample-efficient training of real-robot arm policies from sparse rewards. While contrastive pre-training, data augmentation, demonstrations, and reinforcement learning are alone insufficient for efficient learning, our main contribution is showing that the combination of these disparate techniques results in a simple yet data-efficient method. We show that, given only 10 demonstrations, a single robotic arm can learn sparse-reward manipulation policies from pixels, such as reaching, picking, moving, pulling a large object, flipping a switch, and opening a drawer in just 30 minutes of mean real-world training time. We include videos and code on the project website: https://sites.google.com/view/efficient-robotic-manipulation/home
