生物・ロボットシステムのためのモデルフリー強化学習:理論と実践
Deep Dive into Model-free Reinforcement Learning for Biological and Robotic Systems: Theory and Practice
動物とロボットの身体制御を対象に、モデルフリー強化学習(特にactor-critic法)の数理的・アルゴリズム的側面を解説し、身体性を考慮したフィードバック制御の研究枠組みを提示する。
著者: Yusheng Jiao, Feng Ling, Sina Heydari, Nicolas Heess, Josh Merel, Eva Kanso
分類: cs.RO, cs.AI, cs.LG
原文アブストラクト
Animals and robots exist in a physical world and must coordinate their bodies to achieve behavioral objectives. With recent developments in deep reinforcement learning, it is now possible for scientists and engineers to obtain sensorimotor strategies (policies) for specific tasks using physically simulated bodies and environments. However, the utility of these methods goes beyond the constraints of a specific task; they offer an exciting framework for understanding the organization of an animal sensorimotor system in connection to its morphology and physical interaction with the environment, as well as for deriving general design rules for sensing and actuation in robotic systems. Algorithms and code implementing both learning agents and environments are increasingly available, but the basic assumptions and choices that go into the formulation of an embodied feedback control problem using deep reinforcement learning may not be immediately apparent. Here, we present a concise exposition of the mathematical and algorithmic aspects of model-free reinforcement learning, specifically through the use of \textit{actor-critic} methods, as a tool for investigating the feedback control underlying animal and robotic behavior.
