信号時相論理タスクのための計算可能な強化学習
Tractable Reinforcement Learning of Signal Temporal Logic Objectives
信号時相論理(STL)仕様を満たす方策を強化学習で学ぶ際、状態履歴の必要性による状態空間の指数的増大を、履歴を圧縮した拡張状態表現と目的関数の近似で解決し、性能保証付きで既存手法と比較した論文。
著者: Harish Venkataraman, Derya Aksaray, Peter Seiler
分類: cs.LG, cs.AI, cs.RO, stat.ML
原文アブストラクト
Signal temporal logic (STL) is an expressive language to specify time-bound real-world robotic tasks and safety specifications. Recently, there has been an interest in learning optimal policies to satisfy STL specifications via reinforcement learning (RL). Learning to satisfy STL specifications often needs a sufficient length of state history to compute reward and the next action. The need for history results in exponential state-space growth for the learning problem. Thus the learning problem becomes computationally intractable for most real-world applications. In this paper, we propose a compact means to capture state history in a new augmented state-space representation. An approximation to the objective (maximizing probability of satisfaction) is proposed and solved for in the new augmented state-space. We show the performance bound of the approximate solution and compare it with the solution of an existing technique via simulations.
