日本フィジカルAI新聞

世界のフィジカルAIを、日本語で。

強化学習arXiv:2110.09712

過小評価と過大評価のバランスを取るRealistic Actor-Critic

Balancing Value Underestimation and Overestimation with Realistic Actor-Critic

シェア:XThreadsFacebookLINEはてブBluesky

複数のクリティックの不確実性を利用してQ関数の信頼区間を多様に構築し、過小評価と過大評価のトレードオフを調整するRACを提案。MuJoCoでSAC比10倍のサンプル効率とHumanoidで25%の性能向上を達成した。

著者: Sicen Li, Qinyun Tang, Yiming Pang, Xinmeng Ma, Gang Wang

分類: cs.LG

原文アブストラクト

Model-free deep reinforcement learning (RL) has been successfully applied to challenging continuous control domains. However, poor sample efficiency prevents these methods from being widely used in real-world domains. This paper introduces a novel model-free algorithm, Realistic Actor-Critic(RAC), which can be incorporated with any off-policy RL algorithms to improve sample efficiency. RAC employs Universal Value Function Approximators (UVFA) to simultaneously learn a policy family with the same neural network, each with different trade-offs between underestimation and overestimation. To learn such policies, we introduce uncertainty punished Q-learning, which uses uncertainty from the ensembling of multiple critics to build various confidence-bounds of Q-function. We evaluate RAC on the MuJoCo benchmark, achieving 10x sample efficiency and 25\% performance improvement on the most challenging Humanoid environment compared to SAC.

関連論文

  1. 1Boston Dynamics、車輪付きAtlasの研究試作機を紹介研究・技術
  2. 2newmoとKDDIが資本業務提携 2028年に堺市で自動運転タクシー商用化目指す提携・買収
  3. 3GXOとApptronik、Apolloの物流現場試験を拡大 最大1,000台の展開を構想量産・展開
  4. 4NVIDIA、CoRLでGR00T 2とIsaac Lab 3.0を発表新製品・発表
  5. 5UC BerkeleyとNVIDIAの研究者、AIエージェントでロボット制御コードを仮想環境で改善研究・技術

PR・アフィリエイト広告

MINISFORUM(ミニPC)

MINISFORUM(ミニPC)の公式広告バナーMINISFORUM公式ストアを見る ↗
PR本紙発行元 EmplifAI