深層強化学習による形態と行動のデータ効率的な共適応
Data-efficient Co-Adaptation of Morphology and Behaviour with Deep Reinforcement Learning
Soft Actor-Criticを基に、過去に試した形態と行動の情報を活用して新しい形態の性能を推定し、ロボットの形態と制御器を少ない試作回数で自動的に共適応させる手法を提案した。
著者: Kevin Sebastian Luck, Heni Ben Amor, Roberto Calandra
分類: cs.LG, cs.AI, cs.NE, cs.RO, stat.ML
原文アブストラクト
Humans and animals are capable of quickly learning new behaviours to solve new tasks. Yet, we often forget that they also rely on a highly specialized morphology that co-adapted with motor control throughout thousands of years. Although compelling, the idea of co-adapting morphology and behaviours in robots is often unfeasible because of the long manufacturing times, and the need to re-design an appropriate controller for each morphology. In this paper, we propose a novel approach to automatically and efficiently co-adapt a robot morphology and its controller. Our approach is based on recent advances in deep reinforcement learning, and specifically the soft actor critic algorithm. Key to our approach is the possibility of leveraging previously tested morphologies and behaviors to estimate the performance of new candidate morphologies. As such, we can make full use of the information available for making more informed decisions, with the ultimate goal of achieving a more data-efficient co-adaptation (i.e., reducing the number of morphologies and behaviors tested). Simulated experiments show that our approach requires drastically less design prototypes to find good morphology-behaviour combinations, making this method particularly suitable for future co-adaptation of robot designs in the real world.
