オフライン強化学習によるF1tenth自動運転レース
F1tenth Autonomous Racing With Offline Reinforcement Learning Methods
F1tenthレース環境でオンライン学習が失敗するため、専門家のデモデータを用いたオフライン強化学習を適用し、未見シナリオへのゼロショット転移や環境変化への適応を評価した。
著者: Prajwal Koirala, Cody Fleming
分類: cs.RO, cs.SY, eess.SY
原文アブストラクト
Autonomous racing serves as a critical platform for evaluating automated driving systems and enhancing vehicle mobility intelligence. This work investigates offline reinforcement learning methods to train agents within the dynamic F1tenth racing environment. The study begins by exploring the challenges of online training in the Austria race track environment, where agents consistently fail to complete the laps. Consequently, this research pivots towards an offline strategy, leveraging `expert' demonstration dataset to facilitate agent training. A waypoint-based suboptimal controller is developed to gather data with successful lap episodes. This data is then employed to train offline learning-based algorithms, with a subsequent analysis of the agents' cross-track performance, evaluating their zero-shot transferability from seen to unseen scenarios and their capacity to adapt to changes in environment dynamics. Beyond mere algorithm benchmarking in autonomous racing scenarios, this study also introduces and describes the machinery of our return-conditioned decision tree-based policy, comparing its performance with methods that employ fully connected neural networks, Transformers, and Diffusion Policies and highlighting some insights into method selection for training autonomous agents in driving interactions.
