弾力的な時間ステップによる強化学習
Reinforcement Learning with Elastic Time Steps
制御周波数を動的に調整する弾力的時間ステップを用いた強化学習アルゴリズムMOSEACを提案し、3Dレースゲームでエネルギー効率とタスク性能の向上を実証した。
著者: Dong Wang, Giovanni Beltrame
分類: cs.RO, cs.LG
原文アブストラクト
Traditional Reinforcement Learning (RL) policies are typically implemented with fixed control rates, often disregarding the impact of control rate selection. This can lead to inefficiencies as the optimal control rate varies with task requirements. We propose the Multi-Objective Soft Elastic Actor-Critic (MOSEAC), an off-policy actor-critic algorithm that uses elastic time steps to dynamically adjust the control frequency. This approach minimizes computational resources by selecting the lowest viable frequency. We show that MOSEAC converges and produces stable policies at the theoretical level, and validate our findings in a real-time 3D racing game. MOSEAC significantly outperformed other variable time step approaches in terms of energy efficiency and task effectiveness. Additionally, MOSEAC demonstrated faster and more stable training, showcasing its potential for real-world RL applications in robotics.
