遅延補償の分離:学習されたダイナミクスフィルタリングによる事前学習済みMARLポリシーの強化
Decoupled Delay Compensation: Enhancing Pre-trained MARL Policies via Learned Dynamics Filtering
実世界のマルチエージェント強化学習では通信遅延やパケット損失が性能を劣化させるため、遅延した観測を現在の信念状態推定に置き換えるモジュール型の実行段階状態推定層を提案した。この層は事前学習済みポリシーにプラグインとして追加でき、元の学習アルゴリズムや報酬構造を変更せずに通信遅延やメッセージ損失に対するロバスト性を向上させる。
詳しい要約
1. どんなもの?
2. 先行研究と比べてどこがすごい?
3. 技術・手法の肝は?
4. どうやって有効だと検証した?
5. 議論はある?
6. 次に読むべき論文は?
※ AIが要旨から生成した要約です。正確性は原文をご確認ください。
著者: Maxim Mednikov, Oren Gal
分類: cs.MA, cs.AI, cs.RO
原文アブストラクト
Real-world multi-agent reinforcement learning (MARL) systems must often operate under stale observations, stochastic communication delays, and intermittent packet loss. Policies trained under idealized synchronous conditions frequently exhibit significant performance degradation in these regimes because they act on outdated feedback. We propose a modular execution-stage state-estimation layer that replaces delayed communicated observations with current belief-state estimates. The framework integrates a learned Gated transition model with a recursive Kalman filtering layer to estimate instantaneous states from asynchronous measurements. A primary advantage of this approach is its modularity, The estimator serves as a plug-in for pre-trained policies, requiring no modifications to the original MARL training algorithm, architecture, or reward structure. Evaluation across diverse multi-agent and continuous-control benchmarks demonstrates that the proposed layer consistently enhances robustness to communication latency and message loss. The most significant performance gains are observed in coordination-intensive and dynamically unstable tasks where temporal consistency is critical for control.
