ロバスト安定性保証付きH∞モデルフリー強化学習
$H_\infty$ Model-free Reinforcement Learning with Robust Stability Guarantee
サンプルベースでリアプノフ安定性とロバスト性能を解析し、最大エントロピー法で安定性保証付きの政策を学習する手法を提案した。外乱やパラメータ変動に対して既存手法より高いロバスト性を示す。
著者: Minghao Han, Yuan Tian, Lixian Zhang, Jun Wang, Wei Pan
分類: cs.LG, cs.RO, cs.SY, eess.SY
原文アブストラクト
Reinforcement learning is showing great potentials in robotics applications, including autonomous driving, robot manipulation and locomotion. However, with complex uncertainties in the real-world environment, it is difficult to guarantee the successful generalization and sim-to-real transfer of learned policies theoretically. In this paper, we introduce and extend the idea of robust stability and $H_\infty$ control to design policies with both stability and robustness guarantee. Specifically, a sample-based approach for analyzing the Lyapunov stability and performance robustness of a learning-based control system is proposed. Based on the theoretical results, a maximum entropy algorithm is developed for searching Lyapunov function and designing a policy with provable robust stability guarantee. Without any specific domain knowledge, our method can find a policy that is robust to various uncertainties and generalizes well to different test environments. In our experiments, we show that our method achieves better robustness to both large impulsive disturbances and parametric variations in the environment than the state-of-art results in both robust and generic RL, as well as classic control. Anonymous code is available to reproduce the experimental results at https://github.com/RobustStabilityGuaranteeRL/RobustStabilityGuaranteeRL.
