線形時相論理仕様からのモデルフリー強化学習による制御合成
Control Synthesis from Linear Temporal Logic Specifications using Model-Free Reinforcement Learning
未知の確率的環境において、LTL仕様を満たす確率を最大化する制御方策をモデルフリー強化学習で学習する手法を提案し、報酬と経路依存割引の設計により最適方策への収束を保証した。
著者: Alper Kamil Bozkurt, Yu Wang, Michael M. Zavlanos, Miroslav Pajic
分類: cs.RO, cs.AI, cs.LG
原文アブストラクト
We present a reinforcement learning (RL) framework to synthesize a control policy from a given linear temporal logic (LTL) specification in an unknown stochastic environment that can be modeled as a Markov Decision Process (MDP). Specifically, we learn a policy that maximizes the probability of satisfying the LTL formula without learning the transition probabilities. We introduce a novel rewarding and path-dependent discounting mechanism based on the LTL formula such that (i) an optimal policy maximizing the total discounted reward effectively maximizes the probabilities of satisfying LTL objectives, and (ii) a model-free RL algorithm using these rewards and discount factors is guaranteed to converge to such policy. Finally, we illustrate the applicability of our RL-based synthesis approach on two motion planning case studies.
