リスク感度型逆強化学習のための能動学習
Active Learning for Risk-Sensitive Inverse Reinforcement Learning
人間のリスク選好をモデル化するリスク感度型逆強化学習において、専門家のリスク選好の未知領域を効率的に探索する確率的撹乱サンプリング手法を提案し、収束を加速させる。
著者: Rui Chen, Wenshuo Wang, Zirui Zhao, Ding Zhao
分類: cs.LG, cs.RO, stat.ML
原文アブストラクト
One typical assumption in inverse reinforcement learning (IRL) is that human experts act to optimize the expected utility of a stochastic cost with a fixed distribution. This assumption deviates from actual human behaviors under ambiguity. Risk-sensitive inverse reinforcement learning (RS-IRL) bridges such gap by assuming that humans act according to a random cost with respect to a set of subjectively distorted distributions instead of a fixed one. Such assumption provides the additional flexibility to model human's risk preferences, represented by a risk envelope, in safe-critical tasks. However, like other learning from demonstration techniques, RS-IRL could also suffer inefficient learning due to redundant demonstrations. Inspired by the concept of active learning, this research derives a probabilistic disturbance sampling scheme to enable an RS-IRL agent to query expert support that is likely to expose unrevealed boundaries of the expert's risk envelope. Experimental results confirm that our approach accelerates the convergence of RS-IRL algorithms with lower variance while still guaranteeing unbiased convergence.
