強化学習による最小コスト到達回避問題の解法
Solving Minimum-Cost Reach Avoid using Reinforcement Learning
到達と安全回避の制約下で累積コストを最小化する問題を、Hamilton-Jacobi到達可能性との関連を用いて強化学習で直接解くRC-PPOを提案し、Mujocoベンチマークで最大57%のコスト削減を達成した。
著者: Oswin So, Cheng Ge, Chuchu Fan
分類: cs.LG, cs.RO, math.OC
原文アブストラクト
Current reinforcement-learning methods are unable to directly learn policies that solve the minimum cost reach-avoid problem to minimize cumulative costs subject to the constraints of reaching the goal and avoiding unsafe states, as the structure of this new optimization problem is incompatible with current methods. Instead, a surrogate problem is solved where all objectives are combined with a weighted sum. However, this surrogate objective results in suboptimal policies that do not directly minimize the cumulative cost. In this work, we propose RC-PPO, a reinforcement-learning-based method for solving the minimum-cost reach-avoid problem by using connections to Hamilton-Jacobi reachability. Empirical results demonstrate that RC-PPO learns policies with comparable goal-reaching rates to while achieving up to 57% lower cumulative costs compared to existing methods on a suite of minimum-cost reach-avoid benchmarks on the Mujoco simulator. The project page can be found at https://oswinso.xyz/rcppo.
