深層ガウス過程による逆強化学習
Inverse Reinforcement Learning via Deep Gaussian Process
深層ガウス過程で状態特徴の抽象表現を学習し、最大エントロピー枠組みと組み合わせて少ない実演から複雑な報酬を推定する逆強化学習手法を提案した。
著者: Ming Jin, Andreas Damianou, Pieter Abbeel, Costas Spanos
分類: cs.LG, cs.RO, stat.ML
原文アブストラクト
We propose a new approach to inverse reinforcement learning (IRL) based on the deep Gaussian process (deep GP) model, which is capable of learning complicated reward structures with few demonstrations. Our model stacks multiple latent GP layers to learn abstract representations of the state feature space, which is linked to the demonstrations through the Maximum Entropy learning framework. Incorporating the IRL engine into the nonlinear latent structure renders existing deep GP inference approaches intractable. To tackle this, we develop a non-standard variational approximation framework which extends previous inference schemes. This allows for approximate Bayesian treatment of the feature space and guards against overfitting. Carrying out representation and inverse reinforcement learning simultaneously within our model outperforms state-of-the-art approaches, as we demonstrate with experiments on standard benchmarks ("object world","highway driving") and a new benchmark ("binary world").
