正規化フローによるドメインランダム化でロボットスキルを学習・系列化
Flow-based Domain Randomization for Learning and Sequencing Robotic Skills
正規化フローで環境のサンプリング分布を自動学習するドメインランダム化手法を提案し、6つのシミュレーションと1つの実機タスクで頑健性を検証。さらに学習した分布と特権価値関数を組み合わせ、不確実性を考慮した多段階マニピュレーション計画での分布外検出に応用した。
著者: Aidan Curtis, Eric Li, Michael Noseworthy, Nishad Gothoskar, Sachin Chitta, Hui Li, Leslie Pack Kaelbling, Nicole Carey
分類: cs.RO, cs.AI, cs.LG
原文アブストラクト
Domain randomization in reinforcement learning is an established technique for increasing the robustness of control policies trained in simulation. By randomizing environment properties during training, the learned policy can become robust to uncertainties along the randomized dimensions. While the environment distribution is typically specified by hand, in this paper we investigate automatically discovering a sampling distribution via entropy-regularized reward maximization of a normalizing-flow-based neural sampling distribution. We show that this architecture is more flexible and provides greater robustness than existing approaches that learn simpler, parameterized sampling distributions, as demonstrated in six simulated and one real-world robotics domain. Lastly, we explore how these learned sampling distributions, combined with a privileged value function, can be used for out-of-distribution detection in an uncertainty-aware multi-step manipulation planner.
