速度履歴を用いたSoft Actor-CriticによるIROS'24競技への挑戦
Velocity-History-Based Soft Actor-Critic Tackling IROS'24 Competition "AI Olympics with RealAIGym"
Soft Actor-CriticにCNNで履歴を符号化したコンテキストベクトルを追加し、PendubotとAcrobotの不安定な劣駆動系を安定化する手法を提案した。
著者: Tim Lukas Faust, Habib Maraqten, Erfan Aghadavoodi, Boris Belousov, Jan Peters
分類: cs.RO
原文アブストラクト
The ``AI Olympics with RealAIGym'' competition challenges participants to stabilize chaotic underactuated dynamical systems with advanced control algorithms. In this paper, we present a novel solution submitted to IROS'24 competition, which builds upon Soft Actor-Critic (SAC), a popular model-free entropy-regularized Reinforcement Learning (RL) algorithm. We add a `context' vector to the state, which encodes the immediate history via a Convolutional Neural Network (CNN) to counteract the unmodeled effects on the real system. Our method achieves high performance scores and competitive robustness scores on both tracks of the competition: Pendubot and Acrobot.
