検証可能で合成的な強化学習のためのマルチフィデリティsim-to-realパイプライン
A Multifidelity Sim-to-Real Pipeline for Verifiable and Compositional Reinforcement Learning
複雑なロボットタスクをサブタスクに分解し、マルチフィデリティシミュレーションで検証しながら強化学習ポリシーを訓練・実機展開するフレームワークを提案し、Warthog無人地上ロボットの走行制御で実証した。
著者: Cyrus Neary, Christian Ellis, Aryaman Singh Samyal, Craig Lennon, Ufuk Topcu
分類: cs.RO, cs.AI, cs.SY, eess.SY
原文アブストラクト
We propose and demonstrate a compositional framework for training and verifying reinforcement learning (RL) systems within a multifidelity sim-to-real pipeline, in order to deploy reliable and adaptable RL policies on physical hardware. By decomposing complex robotic tasks into component subtasks and defining mathematical interfaces between them, the framework allows for the independent training and testing of the corresponding subtask policies, while simultaneously providing guarantees on the overall behavior that results from their composition. By verifying the performance of these subtask policies using a multifidelity simulation pipeline, the framework not only allows for efficient RL training, but also for a refinement of the subtasks and their interfaces in response to challenges arising from discrepancies between simulation and reality. In an experimental case study we apply the framework to train and deploy a compositional RL system that successfully pilots a Warthog unmanned ground robot.
