確率的最適制御のための増分サンプリングベースアルゴリズム
An Incremental Sampling-based Algorithm for Stochastic Optimal Control
連続時間・連続状態の確率的最適制御問題に対し、状態空間のランダムサンプリングで有限離散化を逐次生成し、非同期値反復で最適方策を増分的に近似するiMDPを提案した論文。
著者: Vu Anh Huynh, Sertac Karaman, Emilio Frazzoli
分類: cs.RO, cs.SY, eess.SY, math.DS, math.OC, math.PR
原文アブストラクト
In this paper, we consider a class of continuous-time, continuous-space stochastic optimal control problems. Building upon recent advances in Markov chain approximation methods and sampling-based algorithms for deterministic path planning, we propose a novel algorithm called the incremental Markov Decision Process (iMDP) to compute incrementally control policies that approximate arbitrarily well an optimal policy in terms of the expected cost. The main idea behind the algorithm is to generate a sequence of finite discretizations of the original problem through random sampling of the state space. At each iteration, the discretized problem is a Markov Decision Process that serves as an incrementally refined model of the original problem. We show that with probability one, (i) the sequence of the optimal value functions for each of the discretized problems converges uniformly to the optimal value function of the original stochastic optimal control problem, and (ii) the original optimal value function can be computed efficiently in an incremental manner using asynchronous value iterations. Thus, the proposed algorithm provides an anytime approach to the computation of optimal control policies of the continuous problem. The effectiveness of the proposed approach is demonstrated on motion planning and control problems in cluttered environments in the presence of process noise.
関連論文
- 連続時間確率制約付き確率的最適制御における強双対性と双対勾配上昇法確率的最適制御
- パラメータ化入力推論による制約付き確率的最適制御の近似解法確率的最適制御
- 確率的最適制御のための経路積分理論の開拓確率的最適制御
- 近似ベイズ入力推論による効率的な確率的最適制御確率的最適制御
- 確率的最適制御におけるFeynman-Kac前後方向SDEのための価値関数推定器確率的最適制御
- 前向き・後向きRRTによる確率的最適制御確率的最適制御
