実機ロボットでの強化学習アルゴリズムのベンチマーク
Benchmarking Reinforcement Learning Algorithms on Real-World Robots
複数の市販ロボットを用いた実機向け強化学習タスクを提案し、4つの代表的なアルゴリズムの学習性能とハイパーパラメータ感度を実機で評価した。
著者: A. Rupam Mahmood, Dmytro Korenkevych, Gautham Vasan, William Ma, James Bergstra
分類: cs.LG, cs.AI, cs.RO, stat.ML
原文アブストラクト
Through many recent successes in simulation, model-free reinforcement learning has emerged as a promising approach to solving continuous control robotic tasks. The research community is now able to reproduce, analyze and build quickly on these results due to open source implementations of learning algorithms and simulated benchmark tasks. To carry forward these successes to real-world applications, it is crucial to withhold utilizing the unique advantages of simulations that do not transfer to the real world and experiment directly with physical robots. However, reinforcement learning research with physical robots faces substantial resistance due to the lack of benchmark tasks and supporting source code. In this work, we introduce several reinforcement learning tasks with multiple commercially available robots that present varying levels of learning difficulty, setup, and repeatability. On these tasks, we test the learning performance of off-the-shelf implementations of four reinforcement learning algorithms and analyze sensitivity to their hyper-parameters to determine their readiness for applications in various real-world tasks. Our results show that with a careful setup of the task interface and computations, some of these implementations can be readily applicable to physical robots. We find that state-of-the-art learning algorithms are highly sensitive to their hyper-parameters and their relative ordering does not transfer across tasks, indicating the necessity of re-tuning them for each task for best performance. On the other hand, the best hyper-parameter configuration from one task may often result in effective learning on held-out tasks even with different robots, providing a reasonable default. We make the benchmark tasks publicly available to enhance reproducibility in real-world reinforcement learning.
関連論文
- ImagineBench:大規模言語モデルのロールアウトを用いた強化学習の評価ベンチマーク強化学習ベンチマーク
- 記憶・ベンチマーク・ロボット:強化学習で複雑なタスクを解くためのベンチマーク強化学習ベンチマーク
- RobocupGym: RoboCupにおける挑戦的な連続制御ベンチマーク強化学習ベンチマーク
- 視覚ベースロボティクスにおける深層強化学習アルゴリズムのベンチマーク強化学習ベンチマーク
- 教師なし強化学習ベンチマークURLB強化学習ベンチマーク
- dm_control:連続制御のためのソフトウェアとタスク集強化学習ベンチマーク
