ノイズ条件付きエネルギーベースアニーリング報酬（NEAR）：観測からの模倣学習のための生成フレームワーク
Noise-conditioned Energy-based Annealed Rewards (NEAR): A Generative Framework for Imitation Learning from Observation
専門家の動作軌跡（状態のみ）から、エネルギーベース生成モデルとノイズ条件付きスコアマッチングを用いて報酬関数を学習し、強化学習により複雑なロボット運動ポリシーを模倣する手法を提案。
著者: Anish Abhijit Diwan, Julen Urain, Jens Kober, Jan Peters
分類: cs.RO, cs.AI
原文アブストラクト
This paper introduces a new imitation learning framework based on energy-based generative models capable of learning complex, physics-dependent, robot motion policies through state-only expert motion trajectories. Our algorithm, called Noise-conditioned Energy-based Annealed Rewards (NEAR), constructs several perturbed versions of the expert's motion data distribution and learns smooth, and well-defined representations of the data distribution's energy function using denoising score matching. We propose to use these learnt energy functions as reward functions to learn imitation policies via reinforcement learning. We also present a strategy to gradually switch between the learnt energy functions, ensuring that the learnt rewards are always well-defined in the manifold of policy-generated samples. We evaluate our algorithm on complex humanoid tasks such as locomotion and martial arts and compare it with state-only adversarial imitation learning algorithms like Adversarial Motion Priors (AMP). Our framework sidesteps the optimisation challenges of adversarial imitation learning techniques and produces results comparable to AMP in several quantitative metrics across multiple imitation settings.
