少ないほど豊か：マルチタスク強化学習のためのディスパッチャ／エグゼキュータ原理
Less is more -- the Dispatcher/ Executor principle for multi-task Reinforcement Learning
マルチタスク強化学習において、制御器をタスク理解部（ディスパッチャ）とデバイス制御部（エグゼキュータ）に分割し、強い正則化通信路で結ぶ設計原理を提案。データ効率と汎化性能の向上を目指す。
著者: Martin Riedmiller, Andrea Gesmundo, Tim Hertweck, Roland Hafner
分類: cs.LG, cs.AI, cs.RO
原文アブストラクト
Humans instinctively know how to neglect details when it comes to solve complex decision making problems in environments with unforeseeable variations. This abstraction process seems to be a vital property for most biological systems and helps to 'abstract away' unnecessary details and boost generalisation. In this work we introduce the dispatcher/ executor principle for the design of multi-task Reinforcement Learning controllers. It suggests to partition the controller in two entities, one that understands the task (the dispatcher) and one that computes the controls for the specific device (the executor) - and to connect these two by a strongly regularizing communication channel. The core rationale behind this position paper is that changes in structure and design principles can improve generalisation properties and drastically enforce data-efficiency. It is in some sense a 'yes, and ...' response to the current trend of using large neural networks trained on vast amounts of data and bet on emerging generalisation properties. While we agree on the power of scaling - in the sense of Sutton's 'bitter lesson' - we will give some evidence, that considering structure and adding design principles can be a valuable and critical component in particular when data is not abundant and infinite, but is a precious resource.
