ハードウェアをポリシーとして:深層強化学習による機械・計算の協調最適化
Hardware as Policy: Mechanical and Computational Co-Optimization using Deep Reinforcement Learning
ロボットの身体を「ハードウェアポリシー」として微分可能な計算グラフでモデル化し、制御ポリシーと同時に強化学習で最適化する手法を提案。おもちゃの質量-ばね問題と実機の劣駆動ハンド設計で有効性を示した。
著者: Tianjian Chen, Zhanpeng He, Matei Ciocarlie
分類: cs.RO
原文アブストラクト
Deep Reinforcement Learning (RL) has shown great success in learning complex control policies for a variety of applications in robotics. However, in most such cases, the hardware of the robot has been considered immutable, modeled as part of the environment. In this study, we explore the problem of learning hardware and control parameters together in a unified RL framework. To achieve this, we propose to model the robot body as a "hardware policy", analogous to and optimized jointly with its computational counterpart. We show that, by modeling such hardware policies as auto-differentiable computational graphs, the ensuing optimization problem can be solved efficiently by gradient-based algorithms from the Policy Optimization family. We present two such design examples: a toy mass-spring problem, and a real-world problem of designing an underactuated hand. We compare our method against traditional co-optimization approaches, and also demonstrate its effectiveness by building a physical prototype based on the learned hardware parameters. Videos and more details are available at https://roamlab.github.io/hwasp/ .
