俳優を小さくしよう：アクター・クリティック強化学習における小型アクターでの性能維持の事例研究
Honey, I Shrunk The Actor: A Case Study on Preserving Performance with Smaller Actors in Actor-Critic RL
アクターとクリティックのネットワーク構造を独立に設計することで、小型アクターでも同等の性能を維持できることを示し、最大99%のパラメータ削減を達成した。
著者: Siddharth Mysore, Bassel Mabsout, Renato Mancuso, Kate Saenko
分類: cs.LG, cs.RO
原文アブストラクト
Actors and critics in actor-critic reinforcement learning algorithms are functionally separate, yet they often use the same network architectures. This case study explores the performance impact of network sizes when considering actor and critic architectures independently. By relaxing the assumption of architectural symmetry, it is often possible for smaller actors to achieve comparable policy performance to their symmetric counterparts. Our experiments show up to 99% reduction in the number of network weights with an average reduction of 77% over multiple actor-critic algorithms on 9 independent tasks. Given that reducing actor complexity results in a direct reduction of run-time inference cost, we believe configurations of actors and critics are aspects of actor-critic design that deserve to be considered independently, particularly in resource-constrained applications or when deploying multiple actors simultaneously.
