確率的最適制御を近似入力推論として捉える
Stochastic Optimal Control as Approximate Input Inference
推論と制御の双対性に基づき、最適制御を入力分布の推定問題として定式化し、EMとメッセージパッシングで時変線形ガウスフィードバック制御器を導出する手法を提案した。
著者: Joe Watson, Hany Abdulsamad, Jan Peters
分類: cs.LG, cs.RO, cs.SY, eess.SY, stat.ML
原文アブストラクト
Optimal control of stochastic nonlinear dynamical systems is a major challenge in the domain of robot learning. Given the intractability of the global control problem, state-of-the-art algorithms focus on approximate sequential optimization techniques, that heavily rely on heuristics for regularization in order to achieve stable convergence. By building upon the duality between inference and control, we develop the view of Optimal Control as Input Estimation, devising a probabilistic stochastic optimal control formulation that iteratively infers the optimal input distributions by minimizing an upper bound of the control cost. Inference is performed through Expectation Maximization and message passing on a probabilistic graphical model of the dynamical system, and time-varying linear Gaussian feedback controllers are extracted from the joint state-action distribution. This perspective incorporates uncertainty quantification, effective initialization through priors, and the principled regularization inherent to the Bayesian treatment. Moreover, it can be shown that for deterministic linearized systems, our framework derives the maximum entropy linear quadratic optimal control law. We provide a complete and detailed derivation of our probabilistic approach and highlight its advantages in comparison to other deterministic and probabilistic solvers.
