遷移モデルを用いた連続制御方策の学習
Learning Policies for Continuous Control via Transition Models
予測誤差からシステムのダイナミクスを学習し、その遷移モデルを活用して目標位置へ到達する連続制御方策を同時に学習するモジュール型ニューラルネットワークを提案した。
著者: Justus Huebotter, Serge Thill, Marcel van Gerven, Pablo Lanillos
分類: cs.RO, cs.AI, cs.LG, cs.SY, eess.SY
原文アブストラクト
It is doubtful that animals have perfect inverse models of their limbs (e.g., what muscle contraction must be applied to every joint to reach a particular location in space). However, in robot control, moving an arm's end-effector to a target position or along a target trajectory requires accurate forward and inverse models. Here we show that by learning the transition (forward) model from interaction, we can use it to drive the learning of an amortized policy. Hence, we revisit policy optimization in relation to the deep active inference framework and describe a modular neural network architecture that simultaneously learns the system dynamics from prediction errors and the stochastic policy that generates suitable continuous control commands to reach a desired reference position. We evaluated the model by comparing it against the baseline of a linear quadratic regulator, and conclude with additional steps to take toward human-like motor control.
