ロボットマニピュレーションのためのベイズモデルベース強化学習における能動的探索
Active Exploration in Bayesian Model-based Reinforcement Learning for Robot Manipulation
ベイズニューラルネットワークで動的モデルを表現し、情報獲得を最大化する能動的探索を行うことで、ロボットマニピュレーションにおけるモデルベース強化学習のモデル品質とデータ効率を向上させる手法を提案・評価した。
著者: Carlos Plou, Ana C. Murillo, Ruben Martinez-Cantin
分類: cs.RO, cs.LG
原文アブストラクト
Efficiently tackling multiple tasks within complex environment, such as those found in robot manipulation, remains an ongoing challenge in robotics and an opportunity for data-driven solutions, such as reinforcement learning (RL). Model-based RL, by building a dynamic model of the robot, enables data reuse and transfer learning between tasks with the same robot and similar environment. Furthermore, data gathering in robotics is expensive and we must rely on data efficient approaches such as model-based RL, where policy learning is mostly conducted on cheaper simulations based on the learned model. Therefore, the quality of the model is fundamental for the performance of the posterior tasks. In this work, we focus on improving the quality of the model and maintaining the data efficiency by performing active learning of the dynamic model during a preliminary exploration phase based on maximize information gathering. We employ Bayesian neural network models to represent, in a probabilistic way, both the belief and information encoded in the dynamic model during exploration. With our presented strategies we manage to actively estimate the novelty of each transition, using this as the exploration reward. In this work, we compare several Bayesian inference methods for neural networks, some of which have never been used in a robotics context, and evaluate them in a realistic robot manipulation setup. Our experiments show the advantages of our Bayesian model-based RL approach, with similar quality in the results than relevant alternatives with much lower requirements regarding robot execution steps. Unlike related previous studies that focused the validation solely on toy problems, our research takes a step towards more realistic setups, tackling robotic arm end-tasks.
関連論文
- 経験時代における実世界コード・アズ・ポリシー:ACE-CaP-0マニピュレーション
- 把持のアドミタンスによる入出力特性評価:設計パラメータと把持挙動の接続マニピュレーション
- ManiLadder:カテゴリ別・多難易度タスクラダーによるロボットマニピュレーションのベンチマークマニピュレーション
- 成功確率を変える事象を記憶する:Criticゲート付きイベントメモリによる長期的マニピュレーションマニピュレーション
- BiPOP: 長期的な両腕ロボット操作のための部分順序計画マニピュレーション
- RUMI: 拡張現実と触覚フィードバックを備えたRobotiq対応の汎用マニピュレーションインターフェースマニピュレーション
