ドメインランダム化下での学習改善に向けた深層強化学習エージェントの事前学習
Pre-training of Deep RL Agents for Improved Learning under Domain Randomization
ドメインランダム化に不変な知覚エンコーダを事前学習することで、ノイズの多い訓練信号による強化学習の学習阻害を抑え、シミュレーションから実機へのゼロショット転移を改善した。
著者: Artemij Amiranashvili, Max Argus, Lukas Hermann, Wolfram Burgard, Thomas Brox
分類: cs.LG, cs.AI, cs.RO
原文アブストラクト
Visual domain randomization in simulated environments is a widely used method to transfer policies trained in simulation to real robots. However, domain randomization and augmentation hamper the training of a policy. As reinforcement learning struggles with a noisy training signal, this additional nuisance can drastically impede training. For difficult tasks it can even result in complete failure to learn. To overcome this problem we propose to pre-train a perception encoder that already provides an embedding invariant to the randomization. We demonstrate that this yields consistently improved results on a randomized version of DeepMind control suite tasks and a stacking environment on arbitrary backgrounds with zero-shot transfer to a physical robot.
