ガウス過程を用いたマルチフィデリティ強化学習
Multi-Fidelity Reinforcement Learning with Gaussian Processes
複数の忠実度のシミュレータを連鎖させ、ガウス過程を組み込むことで実環境サンプル数を削減する強化学習手法を提案し、地上ロボットのナビゲーションで有効性を示した。
著者: Varun Suryan, Nahush Gondhalekar, Pratap Tokekar
分類: cs.RO
原文アブストラクト
We study the problem of Reinforcement Learning (RL) using as few real-world samples as possible. A naive application of RL can be inefficient in large and continuous state spaces. We present two versions of Multi-Fidelity Reinforcement Learning (MFRL), model-based and model-free, that leverage Gaussian Processes (GPs) to learn the optimal policy in a real-world environment. In the MFRL framework, an agent uses multiple simulators of the real environment to perform actions. With increasing fidelity in a simulator chain, the number of samples used in successively higher simulators can be reduced. By incorporating GPs in the MFRL framework, we empirically observe up to $40\%$ reduction in the number of samples for model-based RL and $60\%$ reduction for the model-free version. We examine the performance of our algorithms through simulations and through real-world experiments for navigation with a ground robot.
