構成経路制御による強化学習ポリシーの安定化
Configuration Path Control
強化学習で得た脆い制御ポリシーを、訓練データと瞬時制御行列推定のみを用いて構成経路空間で事後的に安定化する手法を提案し、平面二足歩行ロボットの外乱実験で2〜4倍の安定性向上を示した。
著者: Sergey Pankov
分類: cs.RO, cs.LG, cs.SY, eess.SY
原文アブストラクト
Reinforcement learning methods often produce brittle policies -- policies that perform well during training, but generalize poorly beyond their direct training experience, thus becoming unstable under small disturbances. To address this issue, we propose a method for stabilizing a control policy in the space of configuration paths. It is applied post-training and relies purely on the data produced during training, as well as on an instantaneous control-matrix estimation. The approach is evaluated empirically on a planar bipedal walker subjected to a variety of perturbations. The control policies obtained via reinforcement learning are compared against their stabilized counterparts. Across different experiments, we find two- to four-fold increase in stability, when measured in terms of the perturbation amplitudes. We also provide a zero-dynamics interpretation of our approach.
