方策探索におけるエントロピックリスク尺度の導入
Entropic Risk Measure in Policy Search
報酬の高次統計を考慮するため、エントロピックリスク尺度を目的関数に組み込んだ方策勾配法を提案し、シミュレーションと実機バドミントンのヒッティング動作学習で評価した。
著者: David Nass, Boris Belousov, Jan Peters
分類: cs.LG, cs.RO
原文アブストラクト
With the increasing pace of automation, modern robotic systems need to act in stochastic, non-stationary, partially observable environments. A range of algorithms for finding parameterized policies that optimize for long-term average performance have been proposed in the past. However, the majority of the proposed approaches does not explicitly take into account the variability of the performance metric, which may lead to finding policies that although performing well on average, can perform spectacularly bad in a particular run or over a period of time. To address this shortcoming, we study an approach to policy optimization that explicitly takes into account higher order statistics of the reward function. In this paper, we extend policy gradient methods to include the entropic risk measure in the objective function and evaluate their performance in simulation experiments and on a real-robot task of learning a hitting motion in robot badminton.
