特徴抽出と方策学習の分離：ゴールベースロボティクスにおける状態表現学習の利点評価
Decoupling feature extraction from policy learning: assessing benefits of state representation learning in goal based robotics
視覚からの強化学習において、状態表現学習を方策学習から切り離す手法を複数評価し、複数の表現を積み重ねる新しい教師なしモデルを提案した。サンプル効率が向上し、ハイパーパラメータ変化にも頑健であることを示した。
著者: Antonin Raffin, Ashley Hill, René Traoré, Timothée Lesort, Natalia Díaz-Rodríguez, David Filliat
分類: cs.LG, cs.RO, stat.ML
原文アブストラクト
Scaling end-to-end reinforcement learning to control real robots from vision presents a series of challenges, in particular in terms of sample efficiency. Against end-to-end learning, state representation learning can help learn a compact, efficient and relevant representation of states that speeds up policy learning, reducing the number of samples needed, and that is easier to interpret. We evaluate several state representation learning methods on goal based robotics tasks and propose a new unsupervised model that stacks representations and combines strengths of several of these approaches. This method encodes all the relevant features, performs on par or better than end-to-end learning with better sample efficiency, and is robust to hyper-parameters change.
関連論文
- 探索的状態表現学習状態表現学習
- ロボティクス事前知識を用いた連続行動空間における低次元状態表現学習状態表現学習
