任意のデモンストレーションによるオフライン模倣学習:ダイナミクス表現の事前学習
Offline Imitation Learning upon Arbitrary Demonstrations by Pre-Training Dynamics Representations
遷移ダイナミクスの分解からダイナミクス表現を事前学習し、限られた専門家データでもオフライン模倣学習を高精度化する手法を提案。非専門家データを再利用でき、MuJoCoや実機四足歩行で少数デモからの学習を実現。
詳しい要約
1. どんなもの?
2. 先行研究と比べてどこがすごい?
3. 技術・手法の肝は?
4. どうやって有効だと検証した?
5. 議論はある?
6. 次に読むべき論文は?
※ AIが要旨から生成した要約です。正確性は原文をご確認ください。
著者: Haitong Ma, Bo Dai, Zhaolin Ren, Yebin Wang, Na Li
分類: cs.RO, cs.LG
原文アブストラクト
Limited data has become a major bottleneck in scaling up offline imitation learning (IL). In this paper, we propose enhancing IL performance under limited expert data by introducing a pre-training stage that learns dynamics representations, derived from factorizations of the transition dynamics. We first theoretically justify that the optimal decision variable of offline IL lies in the representation space, significantly reducing the parameters to learn in the downstream IL. Moreover, the dynamics representations can be learned from arbitrary data collected with the same dynamics, allowing the reuse of massive non-expert data and mitigating the limited data issues. We present a tractable loss function inspired by noise contrastive estimation to learn the dynamics representations at the pre-training stage. Experiments on MuJoCo demonstrate that our proposed algorithm can mimic expert policies with as few as a single trajectory. Experiments on real quadrupeds show that we can leverage pre-trained dynamics representations from simulator data to learn to walk from a few real-world demonstrations.
