ベイズ最適化による因子分解型文脈方策探索
Factored Contextual Policy Search with Bayesian Optimization
文脈を目標型と環境型に分解し、目標型文脈上で経験を直接一般化することで、ベイズ最適化による文脈方策探索のデータ効率を改善する手法を提案した。
著者: Peter Karkus, Andras Kupcsik, David Hsu, Wee Sun Lee
分類: cs.LG, cs.AI, cs.RO, stat.ML
原文アブストラクト
Scarce data is a major challenge to scaling robot learning to truly complex tasks, as we need to generalize locally learned policies over different "contexts". Bayesian optimization approaches to contextual policy search (CPS) offer data-efficient policy learning that generalize over a context space. We propose to improve data-efficiency by factoring typically considered contexts into two components: target-type contexts that correspond to a desired outcome of the learned behavior, e.g. target position for throwing a ball; and environment type contexts that correspond to some state of the environment, e.g. initial ball position or wind speed. Our key observation is that experience can be directly generalized over target-type contexts. Based on that we introduce Factored Contextual Policy Search with Bayesian Optimization for both passive and active learning settings. Preliminary results show faster policy generalization on a simulated toy problem. A full paper extension is available at arXiv:1904.11761
