時相論理仕様に基づく見まね学習の改善
Signal Temporal Logic-Guided Apprenticeship Learning
時相論理で記述されたタスク目標をグラフに符号化し、実演者と学習エージェントの振る舞いを評価する指標を導入することで、必要な実演数を大幅に削減しつつ制御方策を学習する手法を提案した。
著者: Aniruddh G. Puranic, Jyotirmoy V. Deshmukh, Stefanos Nikolaidis
分類: cs.RO, cs.AI, cs.LG
原文アブストラクト
Apprenticeship learning crucially depends on effectively learning rewards, and hence control policies from user demonstrations. Of particular difficulty is the setting where the desired task consists of a number of sub-goals with temporal dependencies. The quality of inferred rewards and hence policies are typically limited by the quality of demonstrations, and poor inference of these can lead to undesirable outcomes. In this letter, we show how temporal logic specifications that describe high level task objectives, are encoded in a graph to define a temporal-based metric that reasons about behaviors of demonstrators and the learner agent to improve the quality of inferred rewards and policies. Through experiments on a diverse set of robot manipulator simulations, we show how our framework overcomes the drawbacks of prior literature by drastically improving the number of demonstrations required to learn a control policy.
