深層強化学習を用いた自律ミニチュアレーシングカーにおける画像ベースの行動平滑化正則化
Image-based Regularization for Action Smoothness in Autonomous Miniature Racing Car with Deep Reinforcement Learning
深層強化学習によるミニチュアカー制御で、画像特徴量に基づく行動平滑化正則化(I-RAS)と適応的な正則化重み制御(IR制御)を提案し、実世界のトラックで成功率と完走率を大幅に改善した。
著者: Hoang-Giang Cao, I Lee, Bo-Jiun Hsu, Zheng-Yi Lee, Yu-Wei Shih, Hsueh-Cheng Wang, I-Chen Wu
分類: cs.RO, cs.SY, eess.SY
原文アブストラクト
Deep reinforcement learning has achieved significant results in low-level controlling tasks. However, for some applications like autonomous driving and drone flying, it is difficult to control behavior stably since the agent may suddenly change its actions which often lowers the controlling system's efficiency, induces excessive mechanical wear, and causes uncontrollable, dangerous behavior to the vehicle. Recently, a method called conditioning for action policy smoothness (CAPS) was proposed to solve the problem of jerkiness in low-dimensional features for applications such as quadrotor drones. To cope with high-dimensional features, this paper proposes image-based regularization for action smoothness (I-RAS) for solving jerky control in autonomous miniature car racing. We also introduce a control based on impact ratio, an adaptive regularization weight to control the smoothness constraint, called IR control. In the experiment, an agent with I-RAS and IR control significantly improves the success rate from 59% to 95%. In the real-world-track experiment, the agent also outperforms other methods, namely reducing the average finish lap time, while also improving the completion rate even without real world training. This is also justified by an agent based on I-RAS winning the 2022 AWS DeepRacer Final Championship Cup.
関連論文
- 安全フィルタリングを超えて:制御バリア関数を活用した連結自動運転車の強化学習強化学習/自動運転
- 運転意図の増幅による計画指向強化学習の性能向上強化学習/自動運転
- 実世界アジャイル運転のためのリセットフリー強化学習:実証研究強化学習/自動運転
- 減衰残差方策最適化による実世界自動運転レースの効率化強化学習/自動運転
- 報酬拡張強化学習による精密自動駐車の連続制御強化学習/自動運転
- 強化学習によるレーシングポリシーの学習強化学習/自動運転
