ユーザーの目的に焦点を当てたロボットのオープンエンド強化学習
Focusing Robot Open-Ended Reinforcement Learning Through Users' Purposes
ユーザーが音声で与えた「目的」を大規模言語モデルで解釈し、目的に関連する物体の近くで探索・報酬生成を行うことで、オープンエンド学習の非効率を抑えるロボットアーキテクチャを提案した。
著者: Emilio Cartoni, Gianluca Cioccolini, Gianluca Baldassarre
分類: cs.RO, cs.LG
原文アブストラクト
Open-Ended Learning (OEL) autonomous robots can acquire new skills and knowledge through direct interaction with their environment, relying on mechanisms such as intrinsic motivations and self-generated goals to guide learning processes. OEL robots are highly relevant for applications as they can autonomously leverage acquired knowledge to perform tasks beneficial to human users in unstructured environments, addressing challenges unforeseen at design time. However, OEL robots face a significant limitation: their openness may lead them to waste time learning information that is irrelevant to tasks desired by specific users. Here, we propose a solution called `Purpose-Directed Open-Ended Learning' (POEL), based on the novel concept of `purpose' introduced in previous work. A purpose specifies what users want the robot to achieve. The key insight of this work is that purpose can focus OEL on learning self-generated classes of tasks that, while unknown during autonomous learning (as typical in OEL), involve objects relevant to the purpose. This concept is operationalised in a novel robot architecture capable of receiving a human purpose through speech-to-text, analysing the scene to identify objects, and using a Large Language Model to reason about which objects are purpose-relevant. These objects are then used to bias OEL exploration towards their spatial proximity and to self-generate rewards that favour interactions with them. The solution is tested in a simulated scenario where a camera-arm-gripper robot interacts freely with purpose-related and distractor objects. For the first time, the results demonstrate the potential advantages of purpose-focused OEL over state-of-the-art OEL methods, enabling robots to handle unstructured environments while steering their learning toward knowledge acquisition relevant to users.
関連論文
- ロボットのオープンエンド学習課題に向けた動機付けアーキテクチャオープンエンド学習
- オープンエンド学習ロボットにおける自律性と整合性の問題:目的フレームワークの形式化オープンエンド学習
- 非定常な相互依存タスクの自律的オープンエンド学習オープンエンド学習
- REAL-X:真にエンドツーエンドな感覚運動自律学習を実現するロボットのオープンエンド学習アーキテクチャオープンエンド学習
- 相互依存タスクの自律的オープンエンド学習オープンエンド学習
