どこでも操作を学習する：強化学習のための視覚的汎化フレームワーク
Learning to Manipulate Anywhere: A Visual Generalizable Framework For Reinforcement Learning
多視点表現学習と空間トランスフォーマー、カリキュラムベースのランダム化を組み合わせ、多様な視覚的擾乱下でも汎化する視覚運動強化学習フレームワークを提案し、実機転移を実証した。
著者: Zhecheng Yuan, Tianming Wei, Shuiqi Cheng, Gu Zhang, Yuanpei Chen, Huazhe Xu
分類: cs.RO, cs.AI, cs.CV
原文アブストラクト
Can we endow visuomotor robots with generalization capabilities to operate in diverse open-world scenarios? In this paper, we propose \textbf{Maniwhere}, a generalizable framework tailored for visual reinforcement learning, enabling the trained robot policies to generalize across a combination of multiple visual disturbance types. Specifically, we introduce a multi-view representation learning approach fused with Spatial Transformer Network (STN) module to capture shared semantic information and correspondences among different viewpoints. In addition, we employ a curriculum-based randomization and augmentation approach to stabilize the RL training process and strengthen the visual generalization ability. To exhibit the effectiveness of Maniwhere, we meticulously design 8 tasks encompassing articulate objects, bi-manual, and dexterous hand manipulation tasks, demonstrating Maniwhere's strong visual generalization and sim2real transfer abilities across 3 hardware platforms. Our experiments show that Maniwhere significantly outperforms existing state-of-the-art methods. Videos are provided at https://gemcollector.github.io/maniwhere/.
