制御可能な物理システム表現の学習に向けて
Towards Learning Controllable Representations of Physical Systems
力学系の表現が制御に適しているかを、真の状態との対応の一意性に基づく2つの指標（時間的滑らかさと相互情報量）で評価し、ペグ挿入タスクの強化学習性能を予測できることを示した論文。
著者: Kevin Haninger, Raul Vicente Garcia, Joerg Krueger
分類: cs.LG, cs.RO, cs.SY, eess.SY
原文アブストラクト
Learned representations of dynamical systems reduce dimensionality, potentially supporting downstream reinforcement learning (RL). However, no established methods predict a representation's suitability for control and evaluation is largely done via downstream RL performance, slowing representation design. Towards a principled evaluation of representations for control, we consider the relationship between the true state and the corresponding representations, proposing that ideally each representation corresponds to a unique true state. This motivates two metrics: temporal smoothness and high mutual information between true state/representation. These metrics are related to established representation objectives, and studied on Lagrangian systems where true state, information requirements, and statistical properties of the state can be formalized for a broad class of systems. These metrics are shown to predict reinforcement learning performance in a simulated peg-in-hole task when comparing variants of autoencoder-based representations.
