自己整合による大規模言語モデルを用いたロボットスキルの報酬学習
Learning Reward for Robot Skills Using Large Language Models via Self-Alignment
LLMが提案した報酬関数を、実行フィードバックに基づく自己整合プロセスで反復的に調整し、人間なしで効率的にロボットスキルの報酬を学習する手法を提案した。
著者: Yuwei Zeng, Yao Mu, Lin Shao
分類: cs.RO, cs.AI
原文アブストラクト
Learning reward functions remains the bottleneck to equip a robot with a broad repertoire of skills. Large Language Models (LLM) contain valuable task-related knowledge that can potentially aid in the learning of reward functions. However, the proposed reward function can be imprecise, thus ineffective which requires to be further grounded with environment information. We proposed a method to learn rewards more efficiently in the absence of humans. Our approach consists of two components: We first use the LLM to propose features and parameterization of the reward, then update the parameters through an iterative self-alignment process. In particular, the process minimizes the ranking inconsistency between the LLM and the learnt reward functions based on the execution feedback. The method was validated on 9 tasks across 2 simulation environments. It demonstrates a consistent improvement over training efficacy and efficiency, meanwhile consuming significantly fewer GPT tokens compared to the alternative mutation-based method.
