不確かなコンテキスト下での安全な強化学習
Safe reinforcement learning in uncertain contexts
コンテキスト変数を直接測定できない場合でも、多クラス分類の頻度論的保証を用いてコンテキストを推定し、安全な学習を実現する手法を提案した。
著者: Dominik Baumann, Thomas B. Schön
分類: cs.LG, cs.RO
原文アブストラクト
When deploying machine learning algorithms in the real world, guaranteeing safety is an essential asset. Existing safe learning approaches typically consider continuous variables, i.e., regression tasks. However, in practice, robotic systems are also subject to discrete, external environmental changes, e.g., having to carry objects of certain weights or operating on frozen, wet, or dry surfaces. Such influences can be modeled as discrete context variables. In the existing literature, such contexts are, if considered, mostly assumed to be known. In this work, we drop this assumption and show how we can perform safe learning when we cannot directly measure the context variables. To achieve this, we derive frequentist guarantees for multi-class classification, allowing us to estimate the current context from measurements. Further, we propose an approach for identifying contexts through experiments. We discuss under which conditions we can retain theoretical guarantees and demonstrate the applicability of our algorithm on a Furuta pendulum with camera measurements of different weights that serve as contexts.
