強化学習制御のための行動方策平滑化正則化
Regularizing Action Policies for Smooth Control with Reinforcement Learning
強化学習で学習した制御方策の行動が滑らかでない問題に対し、CAPSという正則化手法を提案し、実機のドローンで消費電力を約80%削減した。
著者: Siddharth Mysore, Bassel Mabsout, Renato Mancuso, Kate Saenko
分類: cs.RO, cs.LG, cs.SY, eess.SY
原文アブストラクト
A critical problem with the practical utility of controllers trained with deep Reinforcement Learning (RL) is the notable lack of smoothness in the actions learned by the RL policies. This trend often presents itself in the form of control signal oscillation and can result in poor control, high power consumption, and undue system wear. We introduce Conditioning for Action Policy Smoothness (CAPS), an effective yet intuitive regularization on action policies, which offers consistent improvement in the smoothness of the learned state-to-action mappings of neural network controllers, reflected in the elimination of high-frequency components in the control signal. Tested on a real system, improvements in controller smoothness on a quadrotor drone resulted in an almost 80% reduction in power consumption while consistently training flight-worthy controllers. Project website: http://ai.bu.edu/caps
