彩色ネットワークによる強化学習のためのコンパクトアーキテクチャ探索
Reinforcement Learning with Chromatic Networks for Compact Architecture Search
ENASと進化戦略を組み合わせ、エッジの彩色（同重みクラス分割）を探索空間として強化学習ポリシーを極限まで軽量化する手法を提案。17パラメータでも良好な報酬を維持し、従来比6倍の圧縮を実現した。
著者: Xingyou Song, Krzysztof Choromanski, Jack Parker-Holder, Yunhao Tang, Wenbo Gao, Aldo Pacchiano, Tamas Sarlos, Deepali Jain, Yuxiang Yang
分類: cs.NE, cs.AI, cs.LG, cs.RO
原文アブストラクト
We present a neural architecture search algorithm to construct compact reinforcement learning (RL) policies, by combining ENAS and ES in a highly scalable and intuitive way. By defining the combinatorial search space of NAS to be the set of different edge-partitionings (colorings) into same-weight classes, we represent compact architectures via efficient learned edge-partitionings. For several RL tasks, we manage to learn colorings translating to effective policies parameterized by as few as $17$ weight parameters, providing >90% compression over vanilla policies and 6x compression over state-of-the-art compact policies based on Toeplitz matrices, while still maintaining good reward. We believe that our work is one of the first attempts to propose a rigorous approach to training structured neural network architectures for RL problems that are of interest especially in mobile robotics with limited storage and computational resources.
