サンプル効率の良い準最適デモンストレーションからの見習い学習に向けて
Towards Sample-efficient Apprenticeship Learning from Suboptimal Demonstration
準最適なデモンストレーションから報酬関数を学習する際、ノイズ注入ではなく体系的な軌道劣化を用いるS3RRを提案し、従来手法と同等以上の報酬相関を得られることを示した。
著者: Letian Chen, Rohan Paleja, Matthew Gombolay
分類: cs.RO, cs.LG
原文アブストラクト
Learning from Demonstration (LfD) seeks to democratize robotics by enabling non-roboticist end-users to teach robots to perform novel tasks by providing demonstrations. However, as demonstrators are typically non-experts, modern LfD techniques are unable to produce policies much better than the suboptimal demonstration. A previously-proposed framework, SSRR, has shown success in learning from suboptimal demonstration but relies on noise-injected trajectories to infer an idealized reward function. A random approach such as noise-injection to generate trajectories has two key drawbacks: 1) Performance degradation could be random depending on whether the noise is applied to vital states and 2) Noise-injection generated trajectories may have limited suboptimality and therefore will not accurately represent the whole scope of suboptimality. We present Systematic Self-Supervised Reward Regression, S3RR, to investigate systematic alternatives for trajectory degradation. We carry out empirical evaluations and find S3RR can learn comparable or better reward correlation with ground-truth against a state-of-the-art learning from suboptimal demonstration framework.
