共通運用状況の統合によるマルチエージェント協調の強化
Enhancing Multi-Agent Coordination through Common Operating Picture Integration
各エージェントが観測・行動・受信メッセージを統合した「共通運用状況（COP）」を構築・共有することで、分布外の初期状態でも頑健に協調できるマルチエージェント強化学習手法を提案し、StarCraft2で有効性を示した。
著者: Peihong Yu, Bhoram Lee, Aswin Raghavan, Supun Samarasekara, Pratap Tokekar, James Zachary Hare
分類: cs.MA, cs.LG, cs.RO
原文アブストラクト
In multi-agent systems, agents possess only local observations of the environment. Communication between teammates becomes crucial for enhancing coordination. Past research has primarily focused on encoding local information into embedding messages which are unintelligible to humans. We find that using these messages in agent's policy learning leads to brittle policies when tested on out-of-distribution initial states. We present an approach to multi-agent coordination, where each agent is equipped with the capability to integrate its (history of) observations, actions and messages received into a Common Operating Picture (COP) and disseminate the COP. This process takes into account the dynamic nature of the environment and the shared mission. We conducted experiments in the StarCraft2 environment to validate our approach. Our results demonstrate the efficacy of COP integration, and show that COP-based training leads to robust policies compared to state-of-the-art Multi-Agent Reinforcement Learning (MARL) methods when faced with out-of-distribution initial states.
