CoachNet：強化学習のための敵対的サンプリング手法
CoachNet: An Adversarial Sampling Approach for Reinforcement Learning
失敗予測器「CoachNet」をオンライン学習し、失敗確率に基づく確率的サンプリングで苦手なエピソードを優先的に訓練することで、強化学習のサンプル効率とテスト時ロバスト性を向上させる手法を提案。
著者: Elmira Amirloo Abolfathi, Jun Luo, Peyman Yadmellat, Kasra Rezaee
分類: cs.LG, cs.RO
原文アブストラクト
Despite the recent successes of reinforcement learning in games and robotics, it is yet to become broadly practical. Sample efficiency and unreliable performance in rare but challenging scenarios are two of the major obstacles. Drawing inspiration from the effectiveness of deliberate practice for achieving expert-level human performance, we propose a new adversarial sampling approach guided by a failure predictor named "CoachNet". CoachNet is trained online along with the agent to predict the probability of failure. This probability is then used in a stochastic sampling process to guide the agent to more challenging episodes. This way, instead of wasting time on scenarios that the agent has already mastered, training is focused on the agent's "weak spots". We present the design of CoachNet, explain its underlying principles, and empirically demonstrate its effectiveness in improving sample efficiency and test-time robustness in common continuous control tasks.
