強化学習による堅牢なミサイル自動操縦の設計
Reinforcement Learning for Robust Missile Autopilot Design
TRPOを基盤にHERや優先経験再生を改良したSER手法を提案し、ミサイルの縦方向飛行を制御するモデルフリー強化学習エージェントを訓練して最適性能と不確実性への堅牢性を両立させた。
著者: Bernardo Cortez
分類: cs.LG, cs.AI, cs.RO, cs.SY, eess.SY
原文アブストラクト
Designing missiles' autopilot controllers has been a complex task, given the extensive flight envelope and the nonlinear flight dynamics. A solution that can excel both in nominal performance and in robustness to uncertainties is still to be found. While Control Theory often debouches into parameters' scheduling procedures, Reinforcement Learning has presented interesting results in ever more complex tasks, going from videogames to robotic tasks with continuous action domains. However, it still lacks clearer insights on how to find adequate reward functions and exploration strategies. To the best of our knowledge, this work is pioneer in proposing Reinforcement Learning as a framework for flight control. In fact, it aims at training a model-free agent that can control the longitudinal flight of a missile, achieving optimal performance and robustness to uncertainties. To that end, under TRPO's methodology, the collected experience is augmented according to HER, stored in a replay buffer and sampled according to its significance. Not only does this work enhance the concept of prioritized experience replay into BPER, but it also reformulates HER, activating them both only when the training progress converges to suboptimal policies, in what is proposed as the SER methodology. Besides, the Reward Engineering process is carefully detailed. The results show that it is possible both to achieve the optimal performance and to improve the agent's robustness to uncertainties (with low damage on nominal performance) by further training it in non-nominal environments, therefore validating the proposed approach and encouraging future research in this field.
関連論文
- 強化学習による固定翼機の自律迎撃強化学習/飛行制御
- 重心再構成による劣駆動飛行船の二段階強化学習制御強化学習/飛行制御
- AcroRL: 双方向推力を用いたアグレッシブなクアッドローター反転飛行の学習強化学習/飛行制御
- クアッドロータ低レベル制御のための同変強化学習フレームワーク強化学習/飛行制御
- トンボ型タンデム翼実験機のためのConcert強化学習によるリアルタイム制御強化学習/飛行制御
- TD3によるFPVゲートを通過するクアッドコプターのニューラルネットワーク制御強化学習/飛行制御
