事前データからの学習と検索によるスキルベース模倣学習
Learning and Retrieval from Prior Data for Skill-based Imitation Learning
過去のタスクのデータから時間的に拡張されたスキルを抽出し、新規タスク用のポリシーがそれらを呼び出す枠組みを開発。表現学習と検索ベースのデータ拡張で性能を向上させ、模倣学習やオフライン強化学習を上回ることを示した。
著者: Soroush Nasiriany, Tian Gao, Ajay Mandlekar, Yuke Zhu
分類: cs.LG, cs.RO
原文アブストラクト
Imitation learning offers a promising path for robots to learn general-purpose behaviors, but traditionally has exhibited limited scalability due to high data supervision requirements and brittle generalization. Inspired by recent advances in multi-task imitation learning, we investigate the use of prior data from previous tasks to facilitate learning novel tasks in a robust, data-efficient manner. To make effective use of the prior data, the robot must internalize knowledge from past experiences and contextualize this knowledge in novel tasks. To that end, we develop a skill-based imitation learning framework that extracts temporally extended sensorimotor skills from prior data and subsequently learns a policy for the target task that invokes these learned skills. We identify several key design choices that significantly improve performance on novel tasks, namely representation learning objectives to enable more predictable skill representations and a retrieval-based data augmentation mechanism to increase the scope of supervision for policy training. On a collection of simulated and real-world manipulation domains, we demonstrate that our method significantly outperforms existing imitation learning and offline reinforcement learning approaches. Videos and code are available at https://ut-austin-rpl.github.io/sailor
