学習したシステムダイナミクスを用いたニューラル最適制御
Neural Optimal Control using Learned System Dynamics
未知のダイナミクスを持つシステムに対し、ニューラルネットワークで制御器と価値関数を表現し、HJB方程式に基づく損失で学習する手法を提案。学習した遷移関数をHJB方程式に組み込み、フィードバックループで制御信号を生成する。
著者: Selim Engin, Volkan Isler
分類: cs.RO
原文アブストラクト
We study the problem of generating control laws for systems with unknown dynamics. Our approach is to represent the controller and the value function with neural networks, and to train them using loss functions adapted from the Hamilton-Jacobi-Bellman (HJB) equations. In the absence of a known dynamics model, our method first learns the state transitions from data collected by interacting with the system in an offline process. The learned transition function is then integrated to the HJB equations and used to forward simulate the control signals produced by our controller in a feedback loop. In contrast to trajectory optimization methods that optimize the controller for a single initial state, our controller can generate near-optimal control signals for initial states from a large portion of the state space. Compared to recent model-based reinforcement learning algorithms, we show that our method is more sample efficient and trains faster by an order of magnitude. We demonstrate our method in a number of tasks, including the control of a quadrotor with 12 state variables.
