因果的反事実による強化学習のロバスト性向上
Causal Counterfactuals for Improving the Robustness of Reinforcement Learning
因果推論と反事実を組み合わせたCausalCFを提案し、ロボット操作タスクにおける強化学習エージェントのロバスト性を向上させた。
著者: Tom He, Jasmina Gajcin, Ivana Dusparic
分類: cs.LG, cs.AI, cs.RO
原文アブストラクト
Reinforcement learning (RL) is used in various robotic applications. RL enables agents to learn tasks autonomously by interacting with the environment. The more critical the tasks are, the higher the demand for the robustness of the RL systems. Causal RL combines RL and causal inference to make RL more robust. Causal RL agents use a causal representation to capture the invariant causal mechanisms that can be transferred from one task to another. Currently, there is limited research in Causal RL, and existing solutions are usually not complete or feasible for real-world applications. In this work, we propose CausalCF, the first complete Causal RL solution incorporating ideas from Causal Curiosity and CoPhy. Causal Curiosity provides an approach for using interventions, and CoPhy is modified to enable the RL agent to perform counterfactuals. Causal Curiosity has been applied to robotic grasping and manipulation tasks in CausalWorld. CausalWorld provides a realistic simulation environment based on the TriFinger robot. We apply CausalCF to complex robotic tasks and show that it improves the RL agent's robustness using CausalWorld.
