日本フィジカルAI新聞

世界のフィジカルAIを、日本語で。

強化学習arXiv:2102.10456

PPOにおけるクリッピング範囲の減衰手法

Decaying Clipping Range in Proximal Policy Optimization

シェア:XThreadsFacebookLINEはてブBluesky

強化学習アルゴリズムPPOのクリッピング範囲を学習中に線形・指数関数的に減衰させる手法を提案し、序盤の探索と終盤の制約強化を両立させて複数の制御・歩行タスクで有効性を検証した論文。

著者: Mónika Farsang, Luca Szegletes

分類: cs.LG, cs.AI, cs.RO

原文アブストラクト

Proximal Policy Optimization (PPO) is among the most widely used algorithms in reinforcement learning, which achieves state-of-the-art performance in many challenging problems. The keys to its success are the reliable policy updates through the clipping mechanism and the multiple epochs of minibatch updates. The aim of this research is to give new simple but effective alternatives to the former. For this, we propose linearly and exponentially decaying clipping range approaches throughout the training. With these, we would like to provide higher exploration at the beginning and stronger restrictions at the end of the learning phase. We investigate their performance in several classical control and locomotive robotic environments. During the analysis, we found that they influence the achieved rewards and are effective alternatives to the constant clipping method in many reinforcement learning tasks.

関連論文

  1. 1Boston Dynamics、車輪付きAtlasの研究試作機を紹介研究・技術
  2. 2newmoとKDDIが資本業務提携 2028年に堺市で自動運転タクシー商用化目指す提携・買収
  3. 3NVIDIA、CoRLでGR00T 2とIsaac Lab 3.0を発表新製品・発表
  4. 4GXOとApptronik、Apolloの物流現場試験を拡大 最大1,000台の展開を構想量産・展開
  5. 5Google DeepMind、ロボット向けAIを発表新製品・発表

PR・アフィリエイト広告

FRONTIER(GPU搭載BTOパソコン)

FRONTIER(GPU搭載BTOパソコン)の公式広告バナーFRONTIERでGPU搭載PCを見る ↗
PR本紙発行元 EmplifAI