手のひらの中のNeRF：新視点合成によるロボティクスのための補正拡張
NeRF in the Palm of Your Hand: Corrective Augmentation for Robotics via Novel-View Synthesis
NeRFを使って視覚実演に合成ノイズを加え、追加の専門家監督なしにロボット把持方策を改善するオフライン拡張手法SPARTNを提案。
著者: Allan Zhou, Moo Jin Kim, Lirui Wang, Pete Florence, Chelsea Finn
分類: cs.LG, cs.CV, cs.RO
原文アブストラクト
Expert demonstrations are a rich source of supervision for training visual robotic manipulation policies, but imitation learning methods often require either a large number of demonstrations or expensive online expert supervision to learn reactive closed-loop behaviors. In this work, we introduce SPARTN (Synthetic Perturbations for Augmenting Robot Trajectories via NeRF): a fully-offline data augmentation scheme for improving robot policies that use eye-in-hand cameras. Our approach leverages neural radiance fields (NeRFs) to synthetically inject corrective noise into visual demonstrations, using NeRFs to generate perturbed viewpoints while simultaneously calculating the corrective actions. This requires no additional expert supervision or environment interaction, and distills the geometric information in NeRFs into a real-time reactive RGB-only policy. In a simulated 6-DoF visual grasping benchmark, SPARTN improves success rates by 2.8$\times$ over imitation learning without the corrective augmentations and even outperforms some methods that use online supervision. It additionally closes the gap between RGB-only and RGB-D success rates, eliminating the previous need for depth sensors. In real-world 6-DoF robotic grasping experiments from limited human demonstrations, our method improves absolute success rates by $22.5\%$ on average, including objects that are traditionally challenging for depth-based methods. See video results at \url{https://bland.website/spartn}.
