TraKDis: 視覚強化学習のためのTransformerベース知識蒸留と布操作への応用
TraKDis: A Transformer-based Knowledge Distillation Approach for Visual Reinforcement Learning with Application to Cloth Manipulation
特権的な状態情報で訓練した教師エージェントから視覚ベースのエージェントへ知識蒸留する2段階手法を提案し、布折り畳みタスクで既存RLを上回る性能と実機での有効性を示した。
著者: Wei Chen, Nicolas Rojas
分類: cs.RO
原文アブストラクト
Approaching robotic cloth manipulation using reinforcement learning based on visual feedback is appealing as robot perception and control can be learned simultaneously. However, major challenges result due to the intricate dynamics of cloth and the high dimensionality of the corresponding states, what shadows the practicality of the idea. To tackle these issues, we propose TraKDis, a novel Transformer-based Knowledge Distillation approach that decomposes the visual reinforcement learning problem into two distinct stages. In the first stage, a privileged agent is trained, which possesses complete knowledge of the cloth state information. This privileged agent acts as a teacher, providing valuable guidance and training signals for subsequent stages. The second stage involves a knowledge distillation procedure, where the knowledge acquired by the privileged agent is transferred to a vision-based agent by leveraging pre-trained state estimation and weight initialization. TraKDis demonstrates better performance when compared to state-of-the-art RL techniques, showing a higher performance of 21.9%, 13.8%, and 8.3% in cloth folding tasks in simulation. Furthermore, to validate robustness, we evaluate the agent in a noisy environment; the results indicate its ability to handle and adapt to environmental uncertainties effectively. Real robot experiments are also conducted to showcase the efficiency of our method in real-world scenarios.
関連論文
- 経験時代における実世界コード・アズ・ポリシー:ACE-CaP-0マニピュレーション
- 把持のアドミタンスによる入出力特性評価:設計パラメータと把持挙動の接続マニピュレーション
- ManiLadder:カテゴリ別・多難易度タスクラダーによるロボットマニピュレーションのベンチマークマニピュレーション
- 成功確率を変える事象を記憶する:Criticゲート付きイベントメモリによる長期的マニピュレーションマニピュレーション
- BiPOP: 長期的な両腕ロボット操作のための部分順序計画マニピュレーション
- RUMI: 拡張現実と触覚フィードバックを備えたRobotiq対応の汎用マニピュレーションインターフェースマニピュレーション
