SO(2)同変強化学習
$\mathrm{SO}(2)$-Equivariant Reinforcement Learning
最適Q関数と方策の同変・不変性を利用し、回転対称なロボット操作問題向けに同変DQNとSACを提案、サンプル効率の大幅改善を示した。
著者: Dian Wang, Robin Walters, Robert Platt
分類: cs.RO
原文アブストラクト
Equivariant neural networks enforce symmetry within the structure of their convolutional layers, resulting in a substantial improvement in sample efficiency when learning an equivariant or invariant function. Such models are applicable to robotic manipulation learning which can often be formulated as a rotationally symmetric problem. This paper studies equivariant model architectures in the context of $Q$-learning and actor-critic reinforcement learning. We identify equivariant and invariant characteristics of the optimal $Q$-function and the optimal policy and propose equivariant DQN and SAC algorithms that leverage this structure. We present experiments that demonstrate that our equivariant versions of DQN and SAC can be significantly more sample efficient than competing algorithms on an important class of robotic manipulation problems.
