事前学習から熟達へ：分布収縮型強化学習による効率的なスキル習得
From Prior to Pro: Efficient Skill Mastery via Distribution Contractive RL Finetuning
事前学習済みの生成ロボットポリシーを、分布収縮型強化学習（DICE-RL）で微調整し、高成功率の行動を増幅して高性能な「プロ」ポリシーへと進化させる手法を提案。シミュレーションと実機で高次元ピクセル入力からの複雑な長期的操作スキルの習得を実証した。
詳しい要約
1. どんなもの？
2. 先行研究と比べてどこがすごい？
3. 技術・手法の肝は？
4. どうやって有効だと検証した？
5. 議論はある？
6. 次に読むべき論文は？
※ AIが要旨から生成した要約です。正確性は原文をご確認ください。
著者: Zhanyi Sun, Shuran Song
分類: cs.RO, cs.LG
原文アブストラクト
We introduce Distribution Contractive Reinforcement Learning (DICE-RL), a framework that uses reinforcement learning (RL) as a "distribution contraction" operator to refine pretrained generative robot policies. DICE-RL turns a pretrained behavior prior into a high-performing "pro" policy by amplifying high-success behaviors from online feedback. We pretrain a diffusion- or flow-based policy for broad behavioral coverage, then finetune it with a stable, sample-efficient residual off-policy RL framework that combines selective behavior regularization with value-guided action selection. Extensive experiments and analyses show that DICE-RL reliably improves performance with strong stability and sample efficiency. It enables mastery of complex long-horizon manipulation skills directly from high-dimensional pixel inputs, both in simulation and on a real robot. Project website: https://zhanyisun.github.io/dice.rl.2026/.
