2人一般和微分ゲームにおける不連続ナッシュ均衡値の近似
Approximating Discontinuous Nash Equilibrial Values of Two-Player General-Sum Differential Games
報酬が不連続な2人微分ゲームでナッシュ均衡方策を求める際、自己教師あり学習では不連続解を学習できない問題に対し、教師ありナッシュ均衡とHJI PDEを組み合わせたハイブリッド法と報酬を徐々に硬くする値硬化法を比較し、ハイブリッド法が安全性で優れることを示した。
著者: Lei Zhang, Mukesh Ghimire, Wenlong Zhang, Zhe Xu, Yi Ren
分類: cs.LG, cs.GT, cs.RO
原文アブストラクト
Finding Nash equilibrial policies for two-player differential games requires solving Hamilton-Jacobi-Isaacs (HJI) PDEs. Self-supervised learning has been used to approximate solutions of such PDEs while circumventing the curse of dimensionality. However, this method fails to learn discontinuous PDE solutions due to its sampling nature, leading to poor safety performance of the resulting controllers in robotics applications when player rewards are discontinuous. This paper investigates two potential solutions to this problem: a hybrid method that leverages both supervised Nash equilibria and the HJI PDE, and a value-hardening method where a sequence of HJIs are solved with a gradually hardening reward. We compare these solutions using the resulting generalization and safety performance in two vehicle interaction simulation studies with 5D and 9D state spaces, respectively. Results show that with informative supervision (e.g., collision and near-collision demonstrations) and the low cost of self-supervised learning, the hybrid method achieves better safety performance than the supervised, self-supervised, and value hardening approaches on equal computational budget. Value hardening fails to generalize in the higher-dimensional case without informative supervision. Lastly, we show that the neural activation function needs to be continuously differentiable for learning PDEs and its choice can be case dependent.
関連論文
- 逆線形二次ガウスゲーム:制約付き設定と転移可能性ゲーム理論/制御
- 状態制約付き一般和微分ゲームのためのパラメトリック価値近似ゲーム理論/制御
- 制御の不完全性下におけるLQゲームの最適修正フィードバック戦略ゲーム理論/制御
- 近似一般化フィードバックナッシュ均衡の計算ゲーム理論/制御
- 到達可能性ゲームの近似解法ゲーム理論/制御
