Real-to-Sim把持:把持検出におけるシミュレーションと実世界のギャップの再考
Real-to-Sim Grasp: Rethinking the Gap between Simulation and Real World in Grasp Detection
6-DoF把持検出において、実世界の深度画像を推論時にシミュレーション空間へ適応させるReal-to-SimフレームワークR2SGraspを提案し、カメラノイズの影響を回避して高精度な把持検出を実現した。
著者: Jia-Feng Cai, Zibo Chen, Xiao-Ming Wu, Jian-Jian Jiang, Yi-Lin Wei, Wei-Shi Zheng
分類: cs.RO
原文アブストラクト
For 6-DoF grasp detection, simulated data is expandable to train more powerful model, but it faces the challenge of the large gap between simulation and real world. Previous works bridge this gap with a sim-to-real way. However, this way explicitly or implicitly forces the simulated data to adapt to the noisy real data when training grasp detectors, where the positional drift and structural distortion within the camera noise will harm the grasp learning. In this work, we propose a Real-to-Sim framework for 6-DoF Grasp detection, named R2SGrasp, with the key insight of bridging this gap in a real-to-sim way, which directly bypasses the camera noise in grasp detector training through an inference-time real-to-sim adaption. To achieve this real-to-sim adaptation, our R2SGrasp designs the Real-to-Sim Data Repairer (R2SRepairer) to mitigate the camera noise of real depth maps in data-level, and the Real-to-Sim Feature Enhancer (R2SEnhancer) to enhance real features with precise simulated geometric primitives in feature-level. To endow our framework with the generalization ability, we construct a large-scale simulated dataset cost-efficiently to train our grasp detector, which includes 64,000 RGB-D images with 14.4 million grasp annotations. Sufficient experiments show that R2SGrasp is powerful and our real-to-sim perspective is effective. The real-world experiments further show great generalization ability of R2SGrasp. Project page is available on https://isee-laboratory.github.io/R2SGrasp.
関連論文
- 経験時代における実世界コード・アズ・ポリシー:ACE-CaP-0マニピュレーション
- 把持のアドミタンスによる入出力特性評価:設計パラメータと把持挙動の接続マニピュレーション
- ManiLadder:カテゴリ別・多難易度タスクラダーによるロボットマニピュレーションのベンチマークマニピュレーション
- 成功確率を変える事象を記憶する:Criticゲート付きイベントメモリによる長期的マニピュレーションマニピュレーション
- BiPOP: 長期的な両腕ロボット操作のための部分順序計画マニピュレーション
- RUMI: 拡張現実と触覚フィードバックを備えたRobotiq対応の汎用マニピュレーションインターフェースマニピュレーション
