SACHA: ヒューリスティックベース注意機構を備えたSoft Actor-Criticによる部分観測マルチエージェント経路計画
SACHA: Soft Actor-Critic with Heuristic-Based Attention for Partially Observable Multi-Agent Path Finding
部分観測下のマルチエージェント経路計画において、各エージェントが視野内の他エージェントの最短経路ヒューリスティックに注意を向ける注意機構とエージェント中心の批評家を導入し、協調と汎化性能を向上させた強化学習手法。
著者: Qiushi Lin, Hang Ma
分類: cs.RO, cs.AI, cs.MA
原文アブストラクト
Multi-Agent Path Finding (MAPF) is a crucial component for many large-scale robotic systems, where agents must plan their collision-free paths to their given goal positions. Recently, multi-agent reinforcement learning has been introduced to solve the partially observable variant of MAPF by learning a decentralized single-agent policy in a centralized fashion based on each agent's partial observation. However, existing learning-based methods are ineffective in achieving complex multi-agent cooperation, especially in congested environments, due to the non-stationarity of this setting. To tackle this challenge, we propose a multi-agent actor-critic method called Soft Actor-Critic with Heuristic-Based Attention (SACHA), which employs novel heuristic-based attention mechanisms for both the actors and critics to encourage cooperation among agents. SACHA learns a neural network for each agent to selectively pay attention to the shortest path heuristic guidance from multiple agents within its field of view, thereby allowing for more scalable learning of cooperation. SACHA also extends the existing multi-agent actor-critic framework by introducing a novel critic centered on each agent to approximate $Q$-values. Compared to existing methods that use a fully observable critic, our agent-centered multi-agent actor-critic method results in more impartial credit assignment and better generalizability of the learned policy to MAPF instances with varying numbers of agents and types of environments. We also implement SACHA(C), which embeds a communication module in the agent's policy network to enable information exchange among agents. We evaluate both SACHA and SACHA(C) on a variety of MAPF instances and demonstrate decent improvements over several state-of-the-art learning-based MAPF methods with respect to success rate and solution quality.
関連論文
- 異種グラフニューラルネットワークを用いたマルチエージェント経路計画のための共有ロードマップ生成と評価マルチエージェント経路計画
- エレベーター付きマルチエージェント経路探索のための競合ベース探索マルチエージェント経路計画
- 可動障害物を含むマルチエージェント経路探索のための競合ベース探索と優先度計画マルチエージェント経路計画
- 階層型LA-MAPF:大規模エージェントMAPF問題の分解による解法高速化マルチエージェント経路計画
- 自動倉庫における実ロボットダイナミクスと相互依存タスクを考慮したマルチエージェント経路計画マルチエージェント経路計画
- 行き詰まりからの脱出:デッドロック回復アプローチマルチエージェント経路計画
