強化学習による時相論理仕様のロバストな充足
Robust Satisfaction of Temporal Logic Specifications via Reinforcement Learning
未知の確率的ダイナミクスを持つシステムに対し、信号時相論理(STL)式で与えられた時間的に階層化されたタスクを充足するため、充足確率と平均期待ロバストネスを最大化する収束保証付き強化学習アルゴリズムを提案し、ロボットナビゲーションのシミュレーションでロバストネス最大化が確率最大化より優れることを示した。
著者: Austin Jones, Derya Aksaray, Zhaodan Kong, Mac Schwager, Calin Belta
分類: eess.SY, cs.RO, cs.SY
原文アブストラクト
We consider the problem of steering a system with unknown, stochastic dynamics to satisfy a rich, temporally layered task given as a signal temporal logic formula. We represent the system as a Markov decision process in which the states are built from a partition of the state space and the transition probabilities are unknown. We present provably convergent reinforcement learning algorithms to maximize the probability of satisfying a given formula and to maximize the average expected robustness, i.e., a measure of how strongly the formula is satisfied. We demonstrate via a pair of robot navigation simulation case studies that reinforcement learning with robustness maximization performs better than probability maximization in terms of both probability of satisfaction and expected robustness.
