多目的Sim-to-Real学習における単一型とハイブリッド型コントローラの比較
Monolithic vs. hybrid controller for multi-objective Sim-to-Real learning
障害物を避けながら目標に到達する多目的タスクで、単一のモノリシック強化学習コントローラと、単目的コントローラを切り替えるハイブリッド型を比較し、ハイブリッド型の方が学習が容易で成功・失敗のトレードオフが良いことを実機で示した。
著者: Atakan Dag, Alexandre Angleraud, Wenyan Yang, Nataliya Strokina, Roel S. Pieters, Minna Lanz, Joni-Kristian Kamarainen
分類: cs.RO, cs.AI
原文アブストラクト
Simulation to real (Sim-to-Real) is an attractive approach to construct controllers for robotic tasks that are easier to simulate than to analytically solve. Working Sim-to-Real solutions have been demonstrated for tasks with a clear single objective such as "reach the target". Real world applications, however, often consist of multiple simultaneous objectives such as "reach the target" but "avoid obstacles". A straightforward solution in the context of reinforcement learning (RL) is to combine multiple objectives into a multi-term reward function and train a single monolithic controller. Recently, a hybrid solution based on pre-trained single objective controllers and a switching rule between them was proposed. In this work, we compare these two approaches in the multi-objective setting of a robot manipulator to reach a target while avoiding an obstacle. Our findings show that the training of a hybrid controller is easier and obtains a better success-failure trade-off than a monolithic controller. The controllers trained in simulator were verified by a real set-up.
