最大拡散強化学習
Maximum diffusion reinforcement learning
エルゴード過程の統計力学を利用して経験の相関を除去し、1回のタスク試行で学習を可能にする強化学習手法を提案した論文。
著者: Thomas A. Berrueta, Allison Pinosky, Todd D. Murphey
分類: cs.LG, cond-mat.stat-mech, cs.AI, cs.RO
原文アブストラクト
Robots and animals both experience the world through their bodies and senses. Their embodiment constrains their experiences, ensuring they unfold continuously in space and time. As a result, the experiences of embodied agents are intrinsically correlated. Correlations create fundamental challenges for machine learning, as most techniques rely on the assumption that data are independent and identically distributed. In reinforcement learning, where data are directly collected from an agent's sequential experiences, violations of this assumption are often unavoidable. Here, we derive a method that overcomes this issue by exploiting the statistical mechanics of ergodic processes, which we term maximum diffusion reinforcement learning. By decorrelating agent experiences, our approach provably enables single-shot learning in continuous deployments over the course of individual task attempts. Moreover, we prove our approach generalizes well-known maximum entropy techniques, and robustly exceeds state-of-the-art performance across popular benchmarks. Our results at the nexus of physics, learning, and control form a foundation for transparent and reliable decision-making in embodied reinforcement learning agents.
