速度・加速度計測なしで動く物理系のモデルベース強化学習
Model-Based Reinforcement Learning for Physical Systems Without Velocity and Acceleration Measurements
位置計測のみから過去の位置履歴を状態として用い、速度・加速度を必要としない物理に着想を得たカーネル付きガウス過程回帰でモデルを学習し、実機2台のRL制御で有効性を示した。
著者: Alberto Dalla Libera, Diego Romeres, Devesh K. Jha, Bill Yerazunis, Daniel Nikovski
分類: cs.LG, cs.RO, cs.SY, eess.SP, eess.SY, stat.ML
原文アブストラクト
In this paper, we propose a derivative-free model learning framework for Reinforcement Learning (RL) algorithms based on Gaussian Process Regression (GPR). In many mechanical systems, only positions can be measured by the sensing instruments. Then, instead of representing the system state as suggested by the physics with a collection of positions, velocities, and accelerations, we define the state as the set of past position measurements. However, the equation of motions derived by physical first principles cannot be directly applied in this framework, being functions of velocities and accelerations. For this reason, we introduce a novel derivative-free physically-inspired kernel, which can be easily combined with nonparametric derivative-free Gaussian Process models. Tests performed on two real platforms show that the considered state definition combined with the proposed model improves estimation performance and data-efficiency w.r.t. traditional models based on GPR. Finally, we validate the proposed framework by solving two RL control problems for two real robotic systems.
関連論文
- 潜在想像によるパラメータ化行動空間の学習モデルベースRL
- チャンク単位のモデルベース方策最適化モデルベースRL
- モデルベース強化学習における動力学変化下でのリプレイ保持の特性評価モデルベースRL
- モデルベース強化学習のための償却低ランク適応モデルベースRL
- 変化点対応ワールドモデル:動力学シフトの検出と古いリプレイの忘却による回復モデルベースRL
- RENEW: 選好から世界モデルの学習とモデル悪用の修復を目指してモデルベースRL
