SURREAL-System：分散深層強化学習のための完全統合スタック
SURREAL-System: Fully-Integrated Stack for Distributed Deep Reinforcement Learning
ハードウェア抽象化からアルゴリズムまで4層で構成され、ラップトップからクラウドまでスケール可能な分散強化学習フレームワークを提案し、PPOやESを数千コア・数百GPUに拡張してGymとRoboticsで最先端性能を達成した。
著者: Linxi Fan, Yuke Zhu, Jiren Zhu, Zihua Liu, Orien Zeng, Anchit Gupta, Joan Creus-Costa, Silvio Savarese, Li Fei-Fei
分類: cs.LG, cs.RO, stat.ME
原文アブストラクト
We present an overview of SURREAL-System, a reproducible, flexible, and scalable framework for distributed reinforcement learning (RL). The framework consists of a stack of four layers: Provisioner, Orchestrator, Protocol, and Algorithms. The Provisioner abstracts away the machine hardware and node pools across different cloud providers. The Orchestrator provides a unified interface for scheduling and deploying distributed algorithms by high-level description, which is capable of deploying to a wide range of hardware from a personal laptop to full-fledged cloud clusters. The Protocol provides network communication primitives optimized for RL. Finally, the SURREAL algorithms, such as Proximal Policy Optimization (PPO) and Evolution Strategies (ES), can easily scale to 1000s of CPU cores and 100s of GPUs. The learning performances of our distributed algorithms establish new state-of-the-art on OpenAI Gym and Robotics Suites tasks.
