モデル予測制御と強化学習の有限時間リアプノフ指数解析
Finite Time Lyapunov Exponent Analysis of Model Predictive Control and Reinforcement Learning
非定常流れ場中でMPCや強化学習により制御される能動エージェントに対し、有限時間リアプノフ指数(FTLE)を適用し、目標到達コストが類似した領域を分ける構造を可視化・解析した研究。
著者: Kartik Krishna, Steven L. Brunton, Zhuoyuan Song
分類: math.OC, cs.RO, cs.SY, eess.SY, math.DS, nlin.CD
原文アブストラクト
Finite-time Lyapunov exponents (FTLEs) provide a powerful approach to compute time-varying analogs of invariant manifolds in unsteady fluid flow fields. These manifolds are useful to visualize the transport mechanisms of passive tracers advecting with the flow. However, many vehicles and mobile sensors are not passive, but are instead actuated according to some intelligent trajectory planning or control law; for example, model predictive control and reinforcement learning are often used to design energy-efficient trajectories in a dynamically changing background flow. In this work, we investigate the use of FTLE on such controlled agents to gain insight into optimal transport routes for navigation in known unsteady flows. We find that these controlled FTLE (cFTLE) coherent structures separate the flow field into different regions with similar costs of transport to the goal location. These separatrices are functions of the planning algorithm's hyper-parameters, such as the optimization time horizon and the cost of actuation. Computing the invariant sets and manifolds of active agent dynamics in dynamic flow fields is useful in the context of robust motion control, hyperparameter tuning, and determining safe and collision-free trajectories for autonomous systems. Moreover, these cFTLE structures provide insight into effective deployment locations for mobile agents with actuation and energy constraints to traverse the ocean or atmosphere.
