地図ベース経験再生：強化学習における破滅的忘却へのメモリ効率的な解決策
Map-based Experience Replay: A Memory-Efficient Solution to Catastrophic Forgetting in Reinforcement Learning
強化学習の経験再生メモリを自己組織化ネットワークで地図状に整理し、類似サンプルを統合することでメモリを大幅に削減しつつ破滅的忘却を抑える手法を提案した。
著者: Muhammad Burhan Hafez, Tilman Immisch, Tom Weber, Stefan Wermter
分類: cs.LG, cs.AI, cs.RO
原文アブストラクト
Deep Reinforcement Learning agents often suffer from catastrophic forgetting, forgetting previously found solutions in parts of the input space when training on new data. Replay Memories are a common solution to the problem, decorrelating and shuffling old and new training samples. They naively store state transitions as they come in, without regard for redundancy. We introduce a novel cognitive-inspired replay memory approach based on the Grow-When-Required (GWR) self-organizing network, which resembles a map-based mental model of the world. Our approach organizes stored transitions into a concise environment-model-like network of state-nodes and transition-edges, merging similar samples to reduce the memory size and increase pair-wise distance among samples, which increases the relevancy of each sample. Overall, our paper shows that map-based experience replay allows for significant memory reduction with only small performance decreases.
