SPRINT: 言語指示の再ラベリングによるスケーラブルな方策事前学習
SPRINT: Scalable Policy Pre-Training via Language Instruction Relabeling
大規模言語モデルによる指示の再ラベリングとオフライン強化学習によるスキル連鎖で、人手をかけずに多様なスキルを事前学習し、新しい長期的タスクの学習を高速化する手法を提案。
著者: Jesse Zhang, Karl Pertsch, Jiahui Zhang, Joseph J. Lim
分類: cs.RO, cs.AI, cs.LG
原文アブストラクト
Pre-training robot policies with a rich set of skills can substantially accelerate the learning of downstream tasks. Prior works have defined pre-training tasks via natural language instructions, but doing so requires tedious human annotation of hundreds of thousands of instructions. Thus, we propose SPRINT, a scalable offline policy pre-training approach which substantially reduces the human effort needed for pre-training a diverse set of skills. Our method uses two core ideas to automatically expand a base set of pre-training tasks: instruction relabeling via large language models and cross-trajectory skill chaining through offline reinforcement learning. As a result, SPRINT pre-training equips robots with a much richer repertoire of skills. Experimental results in a household simulator and on a real robot kitchen manipulation task show that SPRINT leads to substantially faster learning of new long-horizon tasks than previous pre-training approaches. Website at https://clvrai.com/sprint.
