故障に適応する生存型ロボット制御：ベイズ方策探索と深層強化学習
Survivable Robotic Control through Guided Bayesian Policy Search with Deep Reinforcement Learning
機械的故障で自由度が損なわれても、ベイズ方策勾配とガウス過程の事前分布で効率的に方策を再学習し、六脚ロボットの故障実験でDDPGより少ない試行で高い成功率を達成した。
著者: Sayyed Jaffar Ali Raza, Apan Dastider, Mingjie Lin
分類: cs.RO
原文アブストラクト
Many robot manipulation skills can be represented with deterministic characteristics and there exist efficient techniques for learning parameterized motor plans for those skills. However, one of the active research challenge still remains to sustain manipulation capabilities in situation of a mechanical failure. Ideally, like biological creatures, a robotic agent should be able to reconfigure its control policy by adapting to dynamic adversaries. In this paper, we propose a method that allows an agent to survive in a situation of mechanical loss, and adaptively learn manipulation with compromised degrees of freedom -- we call our method Survivable Robotic Learning (SRL). Our key idea is to leverage Bayesian policy gradient by encoding knowledge bias in posterior estimation, which in turn alleviates future policy search explorations, in terms of sample efficiency and when compared to random exploration based policy search methods. SRL represents policy priors as Gaussian process, which allows tractable computation of approximate posterior (when true gradient is intractable), by incorporating guided bias as proxy from prior replays. We evaluate our proposed method against off-the-shelf model free learning algorithm (DDPG), testing on a hexapod robot platform which encounters incremental failure emulation, and our experiments show that our method improves largely in terms of sample requirement and quantitative success ratio in all failure modes. A demonstration video of our experiments can be viewed at: https://sites.google.com/view/survivalrl
