RTAW: 倉庫環境におけるマルチロボットタスク割り当てのための注意機構に着想を得た強化学習手法
RTAW: An Attention Inspired Reinforcement Learning Method for Multi-Robot Task Allocation in Warehouse Environments
倉庫でのマルチロボットタスク割り当てをマルコフ決定過程として定式化し、注意機構を模したポリシーを持つ深層マルチエージェント強化学習で解く。ロボット数に依存しない埋め込みを用い、PPOで訓練して総移動遅延を最小化し、最大1000台までスケールすることを示した。
著者: Aakriti Agrawal, Amrit Singh Bedi, Dinesh Manocha
分類: cs.RO, cs.MA
原文アブストラクト
We present a novel reinforcement learning based algorithm for multi-robot task allocation problem in warehouse environments. We formulate it as a Markov Decision Process and solve via a novel deep multi-agent reinforcement learning method (called RTAW) with attention inspired policy architecture. Hence, our proposed policy network uses global embeddings that are independent of the number of robots/tasks. We utilize proximal policy optimization algorithm for training and use a carefully designed reward to obtain a converged policy. The converged policy ensures cooperation among different robots to minimize total travel delay (TTD) which ultimately improves the makespan for a sufficiently large task-list. In our extensive experiments, we compare the performance of our RTAW algorithm to state of the art methods such as myopic pickup distance minimization (greedy) and regret based baselines on different navigation schemes. We show an improvement of upto 14% (25-1000 seconds) in TTD on scenarios with hundreds or thousands of tasks for different challenging warehouse layouts and task generation schemes. We also demonstrate the scalability of our approach by showing performance with up to $1000$ robots in simulations.
