QUBO最適化による量子インスパイア型エピソード選択を用いたモンテカルロ強化学習
Quantum-Inspired Episode Selection for Monte Carlo Reinforcement Learning via QUBO Optimization
モンテカルロ強化学習のエピソード選択をQUBO問題として定式化し、量子インスパイア型サンプラーで解くことで、報酬最大化と状態空間カバレッジを両立させ、収束速度と方策品質を改善した。
詳しい要約
1. どんなもの?
2. 先行研究と比べてどこがすごい?
3. 技術・手法の肝は?
4. どうやって有効だと検証した?
5. 議論はある?
6. 次に読むべき論文は?
※ AIが要旨から生成した要約です。正確性は原文をご確認ください。
著者: Hadi Salloum, Ali Jnadi, Yaroslav Kholodov, Alexander Gasnikov
分類: cs.LG, cs.RO
原文アブストラクト
Monte Carlo (MC) reinforcement learning suffers from high sample complexity, especially in environments with sparse rewards, large state spaces, and correlated trajectories. We address these limitations by reformulating episode selection as a Quadratic Unconstrained Binary Optimization (QUBO) problem and solving it with quantum-inspired samplers. Our method, MC+QUBO, integrates a combinatorial filtering step into standard MC policy evaluation: from each batch of trajectories, we select a subset that maximizes cumulative reward while promoting state-space coverage. This selection is encoded as a QUBO, where linear terms favor high-reward episodes and quadratic terms penalize redundancy. We explore both Simulated Quantum Annealing (SQA) and Simulated Bifurcation (SB) as black-box solvers within this framework. Experiments in a finite-horizon GridWorld demonstrate that MC+QUBO outperforms vanilla MC in convergence speed and final policy quality, highlighting the potential of quantum-inspired optimization as a decision-making subroutine in reinforcement learning.
