日本フィジカルAI新聞

世界のフィジカルAIを、日本語で。

強化学習/制御arXiv:2410.16632

連続制御ポリシーにおける高周波振動のベンチマークと低減

Benchmarking Smoothness and Reducing High-Frequency Oscillations in Continuous Control Policies

シェア:XThreadsFacebookLINEはてブBluesky

強化学習ポリシーの高周波振動を抑える損失正則化とアーキテクチャ手法を分類・比較し、実機ロボットを含むベンチマークを実施。両者を組み合わせたハイブリッド手法が制御の滑らかさを26.8%改善しつつ性能低下を2.8%に抑えることを示した。

著者: Guilherme Christmann, Ying-Sheng Luo, Hanjaya Mandala, Wei-Chao Chen

分類: cs.RO, cs.LG

原文アブストラクト

Reinforcement learning (RL) policies are prone to high-frequency oscillations, especially undesirable when deploying to hardware in the real-world. In this paper, we identify, categorize, and compare methods from the literature that aim to mitigate high-frequency oscillations in deep RL. We define two broad classes: loss regularization and architectural methods. At their core, these methods incentivize learning a smooth mapping, such that nearby states in the input space produce nearby actions in the output space. We present benchmarks in terms of policy performance and control smoothness on traditional RL environments from the Gymnasium and a complex manipulation task, as well as three robotics locomotion tasks that include deployment and evaluation with real-world hardware. Finally, we also propose hybrid methods that combine elements from both loss regularization and architectural methods. We find that the best-performing hybrid outperforms other methods, and improves control smoothness by 26.8% over the baseline, with a worst-case performance degradation of just 2.8%.

関連論文

  1. 1Boston Dynamics、車輪付きAtlasの研究試作機を紹介研究・技術
  2. 2newmoとKDDIが資本業務提携 2028年に堺市で自動運転タクシー商用化目指す提携・買収
  3. 3GXOとApptronik、Apolloの物流現場試験を拡大 最大1,000台の展開を構想量産・展開
  4. 4NVIDIA、CoRLでGR00T 2とIsaac Lab 3.0を発表新製品・発表
  5. 5UC BerkeleyとNVIDIAの研究者、AIエージェントでロボット制御コードを仮想環境で改善研究・技術

PR・アフィリエイト広告

FRONTIER(GPU搭載BTOパソコン)

FRONTIER(GPU搭載BTOパソコン)の公式広告バナーFRONTIERでGPU搭載PCを見る ↗
PR本紙発行元 EmplifAI