生の状態軌跡からの能動的報酬機械推論
Active Reward Machine Inference From Raw State Trajectories
報酬機械を生の状態軌跡から直接学習する手法を提案し、報酬やラベル、ノードの観測なしで学習可能な条件を示し、能動学習による効率改善も行う。
詳しい要約
1. どんなもの?
2. 先行研究と比べてどこがすごい?
3. 技術・手法の肝は?
4. どうやって有効だと検証した?
5. 議論はある?
6. 次に読むべき論文は?
※ AIが要旨から生成した要約です。正確性は原文をご確認ください。
著者: Mohamad Louai Shehab, Antoine Aspeel, Necmiye Ozay
分類: cs.RO, cs.AI, cs.FL
原文アブストラクト
Reward machines are automaton-like structures that capture the memory required to accomplish a multi-stage task. When combined with reinforcement learning or optimal control methods, they can be used to synthesize robot policies to achieve such tasks. However, specifying a reward machine by hand, including a labeling function capturing high-level features that the decisions are based on, can be a daunting task. This paper deals with the problem of learning reward machines directly from raw state and policy information. As opposed to existing works, we assume no access to observations of rewards, labels, or machine nodes, and show what trajectory data is sufficient for learning the reward machine in this information-scarce regime. We then extend the result to an active learning setting where we incrementally query trajectory extensions to improve data (and indirectly computational) efficiency. Results are demonstrated with several grid world examples.
