地図ベース経路計画のための強化学習における同変アンサンブルと正則化
Equivariant Ensembles and Regularization for Reinforcement Learning in Map-based Path Planning
強化学習において環境の対称性を活用するため、特殊なネットワーク部品を使わずに同変方策と不変価値関数を構築する同変アンサンブル法を提案し、訓練時の正則化項を加えることで、地図ベース経路計画タスクにおけるサンプル効率と性能を向上させた。
著者: Mirco Theile, Hongpeng Cao, Marco Caccamo, Alberto L. Sangiovanni-Vincentelli
分類: cs.LG, cs.RO
原文アブストラクト
In reinforcement learning (RL), exploiting environmental symmetries can significantly enhance efficiency, robustness, and performance. However, ensuring that the deep RL policy and value networks are respectively equivariant and invariant to exploit these symmetries is a substantial challenge. Related works try to design networks that are equivariant and invariant by construction, limiting them to a very restricted library of components, which in turn hampers the expressiveness of the networks. This paper proposes a method to construct equivariant policies and invariant value functions without specialized neural network components, which we term equivariant ensembles. We further add a regularization term for adding inductive bias during training. In a map-based path planning case study, we show how equivariant ensembles and regularization benefit sample efficiency and performance.
