生成モデルと実世界データを組み合わせたロボット学習のより強力な汎化保証
Stronger Generalization Guarantees for Robot Learning by Combining Generative Models and Real-World Data
生成モデルをポリシーの事前分布として用い、実世界データセットでPAC-Bayes理論に基づき更新することで、未見環境への汎化を保証する枠組みを提案し、ドローン航行と把持タスクで有効性を示した論文。
著者: Abhinav Agarwal, Sushant Veer, Allen Z. Ren, Anirudha Majumdar
分類: cs.RO, cs.LG, cs.SY, eess.SY
原文アブストラクト
We are motivated by the problem of learning policies for robotic systems with rich sensory inputs (e.g., vision) in a manner that allows us to guarantee generalization to environments unseen during training. We provide a framework for providing such generalization guarantees by leveraging a finite dataset of real-world environments in combination with a (potentially inaccurate) generative model of environments. The key idea behind our approach is to utilize the generative model in order to implicitly specify a prior over policies. This prior is updated using the real-world dataset of environments by minimizing an upper bound on the expected cost across novel environments derived via Probably Approximately Correct (PAC)-Bayes generalization theory. We demonstrate our approach on two simulated systems with nonlinear/hybrid dynamics and rich sensing modalities: (i) quadrotor navigation with an onboard vision sensor, and (ii) grasping objects using a depth sensor. Comparisons with prior work demonstrate the ability of our approach to obtain stronger generalization guarantees by utilizing generative models. We also present hardware experiments for validating our bounds for the grasping task.
