マルチンゲール手法と時間一貫性サンプリングアルゴリズムによる確率的最適制御のリスク管理
A Martingale Approach and Time-Consistent Sampling-based Algorithms for Risk Management in Stochastic Optimal Control
リスク制約付き確率的最適制御問題をマルチンゲールで拡張し、サンプリングベースのiMDPアルゴリズムで時間一貫性のある最適フィードバック則を近似的に求める手法を提案した。
著者: Vu Anh Huynh, Leonid Kogan, Emilio Frazzoli
分類: eess.SY, cs.RO, cs.SY, math.DS, math.PR
原文アブストラクト
In this paper, we consider a class of stochastic optimal control problems with risk constraints that are expressed as bounded probabilities of failure for particular initial states. We present here a martingale approach that diffuses a risk constraint into a martingale to construct time-consistent control policies. The martingale stands for the level of risk tolerance over time. By augmenting the system dynamics with the controlled martingale, the original risk-constrained problem is transformed into a stochastic target problem. We extend the incremental Markov Decision Process (iMDP) algorithm to approximate arbitrarily well an optimal feedback policy of the original problem by sampling in the augmented state space and computing proper boundary conditions for the reformulated problem. We show that the algorithm is both probabilistically sound and asymptotically optimal. The performance of the proposed algorithm is demonstrated on motion planning and control problems subject to bounded probability of collision in uncertain cluttered environments.
関連論文
- 連続時間確率制約付き確率的最適制御における強双対性と双対勾配上昇法確率的最適制御
- パラメータ化入力推論による制約付き確率的最適制御の近似解法確率的最適制御
- 確率的最適制御のための経路積分理論の開拓確率的最適制御
- 近似ベイズ入力推論による効率的な確率的最適制御確率的最適制御
- 確率的最適制御におけるFeynman-Kac前後方向SDEのための価値関数推定器確率的最適制御
- 前向き・後向きRRTによる確率的最適制御確率的最適制御
