確率的ダイナミクスモデルによるsim-to-real転送性能の予測
Predicting Sim-to-Real Transfer with Probabilistic Dynamics Models
強化学習ポリシーの実機転送性能を、実機ロールアウトなしで予測する指標を提案。確率的ダイナミクスモデルを実機軌道で評価し、複雑なマニピュレーションタスクで実機性能と高い相関を示した。
著者: Lei M. Zhang, Matthias Plappert, Wojciech Zaremba
分類: cs.LG, cs.AI, cs.RO
原文アブストラクト
We propose a method to predict the sim-to-real transfer performance of RL policies. Our transfer metric simplifies the selection of training setups (such as algorithm, hyperparameters, randomizations) and policies in simulation, without the need for extensive and time-consuming real-world rollouts. A probabilistic dynamics model is trained alongside the policy and evaluated on a fixed set of real-world trajectories to obtain the transfer metric. Experiments show that the transfer metric is highly correlated with policy performance in both simulated and real-world robotic environments for complex manipulation tasks. We further show that the transfer metric can predict the effect of training setups on policy transfer performance.
