ビデオ基盤の最適輸送によるフィードバック効率的なオフライン選好ベース強化学習
Video-Based Optimal Transport for Feedback-Efficient Offline Preference-Based Reinforcement Learning
ビデオ基盤モデルの表現空間で最適輸送を用いて軌道を整列させ、少数のラベルから高品質な擬似ラベルを生成することで、選好ベース強化学習のラベルコストを大幅に削減する半教師ありフレームワークを提案した。
詳しい要約
1. どんなもの?
2. 先行研究と比べてどこがすごい?
3. 技術・手法の肝は?
4. どうやって有効だと検証した?
5. 議論はある?
6. 次に読むべき論文は?
※ AIが要旨から生成した要約です。正確性は原文をご確認ください。
著者: Tung M. Luu, Hwanhee Kim, Younghwan Lee, Chang D. Yoo
分類: cs.RO
原文アブストラクト
Conveying complex objectives to reinforcement learning (RL) agents often requires meticulous reward engineering. Preference-based RL (PbRL) offers a promising alternative by learning reward functions from human feedback, but its scalability is hindered by high labeling costs. Inspired by advances in Video Foundation Models (ViFMs), we present Video-based Optimal Transport Preference (VOTP), a semi-supervised framework that learns effective reward functions from only a handful of labels. By leveraging optimal transport to align visual trajectories within the rich representation space of ViFMs, VOTP effectively generates high-fidelity pseudo-labels for large amounts of unlabeled data, substantially reducing human supervision. Extensive experiments across locomotion and manipulation benchmarks demonstrate the superiority of VOTP, which outperforms state-of-the-art offline PbRL methods under limited feedback budgets. We also showcase the robustness of VOTP in the presence of visual distractors and validate its utility on real robotic tasks, where it learns meaningful rewards with minimal human input.
