物理アタリ:ロボット上でのリアルタイム強化学習のための堅牢でアクセスしやすいプラットフォーム
Physical Atari: A Robust and Accessible Platform for Real-time Reinforcement Learning on Robots
Atariコントローラを物理的に操作するロボットとゲーム画面を表示するデバイスを組み合わせ、実世界で強化学習アルゴリズムを研究できるシステム「Physical Atari」を構築した。低コストで堅牢な設計により、ロボット上での直接学習と適応の重要性を示した。
詳しい要約
1. どんなもの?
2. 先行研究と比べてどこがすごい?
3. 技術・手法の肝は?
4. どうやって有効だと検証した?
5. 議論はある?
6. 次に読むべき論文は?
※ AIが要旨から生成した要約です。正確性は原文をご確認ください。
著者: Khurram Javed, Joseph Modayil, Gloria Kennickell, Richard S. Sutton, John Carmack
分類: cs.RO, cs.AI
原文アブストラクト
We built a robot called the Robotroller that actuates an Atari CX40+ controller and a device called the Atari Devbox that renders the game frame and the reward signal from the Arcade Learning Environment on a screen. The Robotroller and the Atari Devbox, together with an off-the-shelf camera and a desktop computer, constitute a system that can be used to study reinforcement learning algorithms in the physical world. We call the full system Physical Atari. In this paper, we detail the key decisions that make Physical Atari a robust and accessible platform. To make the system robust, we designed the Robotroller so that all movement is done through bearings, which reduces wear. Additionally, we wrote software that monitors the state of the servos at a high frequency and intervenes to limit stress. To make the system accessible, we used affordable off-the-shelf components and parts that can be manufactured using consumer 3D printers. Physical Atari can be built for under $1,000 and has been used for weeks of non-stop reinforcement learning experiments without any mechanical failures. We used it to validate that reinforcement learning algorithms can learn directly on robots and show that even small distribution shifts between learning and deployment can significantly degrade the performance of policies. Our results underscore the importance of on-device adaptation for strong performance on robots.
