安全強化学習における長期的安全性と不確実性への対処
Handling Long-Term Safety and Uncertainty in Safe Reinforcement Learning
モデルベースの安全強化学習手法ATACOMを学習可能な制約で拡張し、長期的な安全性と不確実性に対処できるようにした研究。
著者: Jonas Günster, Puze Liu, Jan Peters, Davide Tateo
分類: cs.LG, cs.RO
原文アブストラクト
Safety is one of the key issues preventing the deployment of reinforcement learning techniques in real-world robots. While most approaches in the Safe Reinforcement Learning area do not require prior knowledge of constraints and robot kinematics and rely solely on data, it is often difficult to deploy them in complex real-world settings. Instead, model-based approaches that incorporate prior knowledge of the constraints and dynamics into the learning framework have proven capable of deploying the learning algorithm directly on the real robot. Unfortunately, while an approximated model of the robot dynamics is often available, the safety constraints are task-specific and hard to obtain: they may be too complicated to encode analytically, too expensive to compute, or it may be difficult to envision a priori the long-term safety requirements. In this paper, we bridge this gap by extending the safe exploration method, ATACOM, with learnable constraints, with a particular focus on ensuring long-term safety and handling of uncertainty. Our approach is competitive or superior to state-of-the-art methods in final performance while maintaining safer behavior during training.
