安全な最適制御におけるタスク論理のためのベルマン価値分解
Bellman Value Decomposition for Task Logic in Safe Optimal Control
時相論理で記述された複雑なタスクのベルマン価値を、到達回避・回避・新たに提案する到達回避ループの各ベルマン方程式で結ばれたグラフに分解し、それを二層ニューラルネットに埋め込むVDPPOを提案。高次元タスクで安全性と活性を自動的に両立させ、既存手法を大きく上回る性能を示した。
詳しい要約
1. どんなもの？
2. 先行研究と比べてどこがすごい？
3. 技術・手法の肝は？
4. どうやって有効だと検証した？
5. 議論はある？
6. 次に読むべき論文は？
※ AIが要旨から生成した要約です。正確性は原文をご確認ください。
著者: William Sharpless, Oswin So, Dylan Hirsch, Sylvia Herbert, Chuchu Fan
分類: cs.RO, cs.SY, eess.SY
原文アブストラクト
Real-world tasks involve nuanced combinations of goal and safety specifications. In high dimensions, the challenge is exacerbated: formal automata become cumbersome, and the combination of sparse rewards tends to require laborious tuning. In this work, we consider the innate structure of the Bellman Value as a means to naturally organize the problem for improved automatic performance. Namely, we prove the Bellman Value for a complex task defined in temporal logic can be decomposed into a graph of Bellman Values, connected by a set of well-known Bellman equations (BEs): the Reach-Avoid BE, the Avoid BE, and a novel type, the Reach-Avoid-Loop BE. To solve the Value and optimal policy, we propose VDPPO, which embeds the decomposed Value graph into a two-layer neural net, bootstrapping the implicit dependencies. We conduct a variety of simulated and hardware experiments to test our method on complex, high-dimensional tasks involving heterogeneous teams and nonlinear dynamics. Ultimately, we find this approach greatly improves performance over existing baselines, balancing safety and liveness automatically.
