マルチエージェント強化学習のための視覚的通信マップ
A Visual Communication Map for Multi-Agent Deep Reinforcement Learning
エージェントの状態を視覚的なマップで表現し、共有パラメータネットワークとA3Cで多数・異種エージェントの協調学習を可能にする手法を提案した。
著者: Ngoc Duy Nguyen, Thanh Thi Nguyen, Doug Creighton, Saeid Nahavandi
分類: cs.LG, cs.AI, cs.GT, cs.MA, cs.RO
原文アブストラクト
Deep reinforcement learning has been applied successfully to solve various real-world problems and the number of its applications in the multi-agent settings has been increasing. Multi-agent learning distinctly poses significant challenges in the effort to allocate a concealed communication medium. Agents receive thorough knowledge from the medium to determine subsequent actions in a distributed nature. Apparently, the goal is to leverage the cooperation of multiple agents to achieve a designated objective efficiently. Recent studies typically combine a specialized neural network with reinforcement learning to enable communication between agents. This approach, however, limits the number of agents or necessitates the homogeneity of the system. In this paper, we have proposed a more scalable approach that not only deals with a great number of agents but also enables collaboration between dissimilar functional agents and compatibly combined with any deep reinforcement learning methods. Specifically, we create a global communication map to represent the status of each agent in the system visually. The visual map and the environmental state are fed to a shared-parameter network to train multiple agents concurrently. Finally, we select the Asynchronous Advantage Actor-Critic (A3C) algorithm to demonstrate our proposed scheme, namely Visual communication map for Multi-agent A3C (VMA3C). Simulation results show that the use of visual communication map improves the performance of A3C regarding learning speed, reward achievement, and robustness in multi-agent problems.
関連論文
- GARDEN: 証拠ナビゲーションによる汎用自律報酬設計マルチエージェント強化学習
- サポート保持蒸留によるマルチエージェント協調マルチエージェント強化学習
- 未来の協力者と協調する:参加タイミングがずれるマルチエージェント強化学習マルチエージェント強化学習
- 部分的観測動的ゲームにおける未知の対戦相手に対するレベルK政策の編成マルチエージェント強化学習
- 置換ロバスト性だけでは不十分:マルチエージェントTransformer方策における行動崩壊マルチエージェント強化学習
- テスト時マルチエージェント協調のための分解価値勾配フローマルチエージェント強化学習
