適応的ポテンシャル関数による連続ロボットリーチングタスクでの強化学習アルゴリズムの強化
Boosting Reinforcement Learning Algorithms in Continuous Robotic Reaching Tasks using Adaptive Potential Functions
連続行動空間の実ロボットリーチングタスクにおいて、適応的ポテンシャル関数とDDPGを組み合わせたAPF-DDPGを提案し、学習速度とロバスト性でDDPGを上回ることを示した。
著者: Yifei Chen, Lambert Schomaker, Francisco Cruz
分類: cs.RO
原文アブストラクト
In reinforcement learning, reward shaping is an efficient way to guide the learning process of an agent, as the reward can indicate the optimal policy of the task. The potential-based reward shaping framework was proposed to guarantee policy invariance after reward shaping, where a potential function is used to calculate the shaping reward. In former work, we proposed a novel adaptive potential function (APF) method to learn the potential function concurrently with training the agent based on information collected by the agent during the training process, and examined the APF method in discrete action space scenarios. This paper investigates the feasibility of using APF in solving continuous-reaching tasks in a real-world robotic scenario with continuous action space. We combine the Deep Deterministic Policy Gradient (DDPG) algorithm and our proposed method to form a new algorithm called APF-DDPG. To compare APF-DDPG with DDPG, we designed a task where the agent learns to control Baxter's right arm to reach a goal position. The experimental results show that the APF-DDPG algorithm outperforms the DDPG algorithm on both learning speed and robustness.
