拡散ポリシーと基盤モデルによるロボットスキル学習システム
A Robotic Skill Learning System Built Upon Diffusion Policies and Foundation Models
視覚運動拡散ポリシーと大規模マルチモーダル基盤モデルを組み合わせ、自然言語指示によるスキル選択と事前条件チェックを行うロボットスキル学習システムを構築し、実世界の配膳タスクで実証した。
著者: Nils Ingelhag, Jesper Munkeby, Jonne van Haastregt, Anastasia Varava, Michael C. Welle, Danica Kragic
分類: cs.RO
原文アブストラクト
In this paper, we build upon two major recent developments in the field, Diffusion Policies for visuomotor manipulation and large pre-trained multimodal foundational models to obtain a robotic skill learning system. The system can obtain new skills via the behavioral cloning approach of visuomotor diffusion policies given teleoperated demonstrations. Foundational models are being used to perform skill selection given the user's prompt in natural language. Before executing a skill the foundational model performs a precondition check given an observation of the workspace. We compare the performance of different foundational models to this end as well as give a detailed experimental evaluation of the skills taught by the user in simulation and the real world. Finally, we showcase the combined system on a challenging food serving scenario in the real world. Videos of all experimental executions, as well as the process of teaching new skills in simulation and the real world, are available on the project's website.
