安全制約を埋め込んだMDPと軌道最適化による強化学習
Reinforcement Learning in a Safety-Embedded MDP with Trajectory Optimization
安全制約をMDPの行動空間に埋め込み、軌道最適化で安全な軌道に変換することで、報酬最大化と安全性の両立を図る強化学習手法を提案し、実機の箱押しタスクで有効性を示した。
著者: Fan Yang, Wenxuan Zhou, Zuxin Liu, Ding Zhao, David Held
分類: cs.RO, cs.AI
原文アブストラクト
Safe Reinforcement Learning (RL) plays an important role in applying RL algorithms to safety-critical real-world applications, addressing the trade-off between maximizing rewards and adhering to safety constraints. This work introduces a novel approach that combines RL with trajectory optimization to manage this trade-off effectively. Our approach embeds safety constraints within the action space of a modified Markov Decision Process (MDP). The RL agent produces a sequence of actions that are transformed into safe trajectories by a trajectory optimizer, thereby effectively ensuring safety and increasing training stability. This novel approach excels in its performance on challenging Safety Gym tasks, achieving significantly higher rewards and near-zero safety violations during inference. The method's real-world applicability is demonstrated through a safe and effective deployment in a real robot task of box-pushing around obstacles.
