潜在予測エンパワーメント：シミュレータなしでエンパワーメントを測る
Latent-Predictive Empowerment: Measuring Empowerment without a Simulator
環境の完全なシミュレータを必要とせず、潜在予測モデルのみでエンパワーメントを計算し、多様なスキルセットを学習する手法を提案した。
著者: Andrew Levy, Alessandro Allievi, George Konidaris
分類: cs.AI, cs.LG, cs.RO
原文アブストラクト
Empowerment has the potential to help agents learn large skillsets, but is not yet a scalable solution for training general-purpose agents. Recent empowerment methods learn diverse skillsets by maximizing the mutual information between skills and states; however, these approaches require a model of the transition dynamics, which can be challenging to learn in realistic settings with high-dimensional and stochastic observations. We present Latent-Predictive Empowerment (LPE), an algorithm that can compute empowerment in a more practical manner. LPE learns large skillsets by maximizing an objective that is a principled replacement for the mutual information between skills and states and that only requires a simpler latent-predictive model rather than a full simulator of the environment. We show empirically in a variety of settings--including ones with high-dimensional observations and highly stochastic transition dynamics--that our empowerment objective (i) learns similar-sized skillsets as the leading empowerment algorithm that assumes access to a model of the transition dynamics and (ii) outperforms other model-based approaches to empowerment.
