実世界運転のための物理情報に基づく模倣強化学習
Physics-informed Imitative Reinforcement Learning for Real-world Driving
専門家のデモと探索データを併用し、車両ダイナミクスの物理原理を学習過程から自然に獲得する物理情報付き模倣強化学習を提案し、Waymaxベンチマークで衝突率と路外率を大幅に削減した。
著者: Hang Zhou, Yihao Qin, Dan Xu, Yiding Ji
分類: cs.RO, cs.AI, cs.LG
原文アブストラクト
Recent advances in imitative reinforcement learning (IRL) have considerably enhanced the ability of autonomous agents to assimilate expert demonstrations, leading to rapid skill acquisition in a range of demanding tasks. However, such learning-based agents face significant challenges when transferring knowledge to highly dynamic closed-loop environments. Their performance is significantly impacted by the conflicting optimization objectives of imitation learning (IL) and reinforcement learning (RL), sample inefficiency, and the complexity of uncovering the hidden world model and physics. To address this challenge, we propose a physics-informed IRL that is entirely data-driven. It leverages both expert demonstration data and exploratory data with a joint optimization objective, allowing the underlying physical principles of vehicle dynamics to emerge naturally from the training process. The performance is evaluated through empirical experiments and results exceed popular IL, RL and IRL algorithms in closed-loop settings on Waymax benchmark. Our approach exhibits 37.8% reduction in collision rate and 22.2% reduction in off-road rate compared to the baseline method.
