不確かな区分アフィン系に対する非凸制約下でのロバスト・アクション・ガバナと安全強化学習
Robust Action Governor for Uncertain Piecewise Affine Systems with Non-convex Constraints and Safe Reinforcement Learning
パラメータ不確かさと加法性不確かさを持つ区分アフィン系に対し、非凸制約を満たすロバスト・アクション・ガバナを提案し、強化学習の探索中も安全性を保証する手法を開発した。
著者: Yutong Li, Nan Li, H. Eric Tseng, Anouck Girard, Dimitar Filev, Ilya Kolmanovsky
分類: eess.SY, cs.AI, cs.RO, cs.SY
原文アブストラクト
The action governor is an add-on scheme to a nominal control loop that monitors and adjusts the control actions to enforce safety specifications expressed as pointwise-in-time state and control constraints. In this paper, we introduce the Robust Action Governor (RAG) for systems the dynamics of which can be represented using discrete-time Piecewise Affine (PWA) models with both parametric and additive uncertainties and subject to non-convex constraints. We develop the theoretical properties and computational approaches for the RAG. After that, we introduce the use of the RAG for realizing safe Reinforcement Learning (RL), i.e., ensuring all-time constraint satisfaction during online RL exploration-and-exploitation process. This development enables safe real-time evolution of the control policy and adaptation to changes in the operating environment and system parameters (due to aging, damage, etc.). We illustrate the effectiveness of the RAG in constraint enforcement and safe RL using the RAG by considering their applications to a soft-landing problem of a mass-spring-damper system.
