進捗に応じた選択的クエリで人間参加型強化学習を効率化
Selective Progress-Aware Querying for Human-in-the-Loop Reinforcement Learning
学習の停滞や悪化を検知したときだけ人間にフィードバックを求めるクエリ方策SPARQを提案し、常時フィードバックと同等の成功率を約半分のフィードバック量で達成した。
詳しい要約
1. どんなもの?
2. 先行研究と比べてどこがすごい?
3. 技術・手法の肝は?
4. どうやって有効だと検証した?
5. 議論はある?
6. 次に読むべき論文は?
※ AIが要旨から生成した要約です。正確性は原文をご確認ください。
著者: Anujith Muraleedharan, Anamika J H
分類: cs.RO
原文アブストラクト
Human feedback can greatly accelerate robot learning, but in real-world settings, such feedback is costly and limited. Existing human-in-the-loop reinforcement learning (HiL-RL) methods often assume abundant feedback, limiting their practicality for physical robot deployment. In this work, we introduce SPARQ, a progress-aware query policy that requests feedback only when learning stagnates or worsens, thereby reducing unnecessary oracle calls. We evaluate SPARQ on a simulated UR5 cube-picking task in PyBullet, comparing against three baselines: no feedback, random querying, and always querying. Our experiments show that SPARQ achieves near-perfect task success, matching the performance of always querying while consuming about half the feedback budget. It also provides more stable and efficient learning than random querying, and significantly improves over training without feedback. These findings suggest that selective, progress-based query strategies can make HiL-RL more efficient and scalable for robots operating under realistic human effort constraints.
関連論文
- 実世界のHuman-in-the-Loop深層強化学習を設計するための体系的手法:顕著な特徴、課題、トレードオフHuman-in-the-Loop強化学習
