ベイジアンソフトアクター・クリティック:有向非巡回戦略グラフに基づく深層強化学習
Bayesian Soft Actor-Critic: A Directed Acyclic Strategy Graph Based Deep Reinforcement Learning
複雑な方策をベイジアン連鎖で複数の単純なサブ方策に分解し、ベイジアン戦略ネットワークとして統合するBSACを提案し、連続制御ベンチマークで学習効率の向上を示した。
著者: Qin Yang, Ramviyas Parasuraman
分類: cs.AI, cs.LG, cs.RO
原文アブストラクト
Adopting reasonable strategies is challenging but crucial for an intelligent agent with limited resources working in hazardous, unstructured, and dynamic environments to improve the system's utility, decrease the overall cost, and increase mission success probability. This paper proposes a novel directed acyclic strategy graph decomposition approach based on Bayesian chaining to separate an intricate policy into several simple sub-policies and organize their relationships as Bayesian strategy networks (BSN). We integrate this approach into the state-of-the-art DRL method -- soft actor-critic (SAC), and build the corresponding Bayesian soft actor-critic (BSAC) model by organizing several sub-policies as a joint policy. We compare our method against the state-of-the-art deep reinforcement learning algorithms on the standard continuous control benchmarks in the OpenAI Gym environment. The results demonstrate that the promising potential of the BSAC method significantly improves training efficiency.
