仮想記憶で安定して運転を学習する
Steadily Learn to Drive with Virtual Memory
高次元の画像入力から潜在状態と潜在ダイナミクスモデルを学習し、想像上の軌跡を仮想記憶として方策学習に用いることで、データ効率と学習安定性を高めた自動運転向け強化学習手法を提案した。
著者: Yuhang Zhang, Yao Mu, Yujie Yang, Yang Guan, Shengbo Eben Li, Qi Sun, Jianyu Chen
分類: cs.LG, cs.AI, cs.RO, cs.SY, eess.SY
原文アブストラクト
Reinforcement learning has shown great potential in developing high-level autonomous driving. However, for high-dimensional tasks, current RL methods suffer from low data efficiency and oscillation in the training process. This paper proposes an algorithm called Learn to drive with Virtual Memory (LVM) to overcome these problems. LVM compresses the high-dimensional information into compact latent states and learns a latent dynamic model to summarize the agent's experience. Various imagined latent trajectories are generated as virtual memory by the latent dynamic model. The policy is learned by propagating gradient through the learned latent model with the imagined latent trajectories and thus leads to high data efficiency. Furthermore, a double critic structure is designed to reduce the oscillation during the training process. The effectiveness of LVM is demonstrated by an image-input autonomous driving task, in which LVM outperforms the existing method in terms of data efficiency, learning stability, and control performance.
