SpawnNet: 事前学習済みネットワークから汎化可能な視覚運動スキルを学習する
SpawnNet: Learning Generalizable Visuomotor Skills from Pre-trained Networks
凍結した事前学習済み視覚バックボーンのボトルネックを解消するため、多層表現を別ネットワークに融合する二流アーキテクチャSpawnNetを提案し、模倣学習におけるカテゴリ汎化性能を向上させた。
著者: Xingyu Lin, John So, Sashwat Mahalingam, Fangchen Liu, Pieter Abbeel
分類: cs.RO, cs.CV
原文アブストラクト
The existing internet-scale image and video datasets cover a wide range of everyday objects and tasks, bringing the potential of learning policies that generalize in diverse scenarios. Prior works have explored visual pre-training with different self-supervised objectives. Still, the generalization capabilities of the learned policies and the advantages over well-tuned baselines remain unclear from prior studies. In this work, we present a focused study of the generalization capabilities of the pre-trained visual representations at the categorical level. We identify the key bottleneck in using a frozen pre-trained visual backbone for policy learning and then propose SpawnNet, a novel two-stream architecture that learns to fuse pre-trained multi-layer representations into a separate network to learn a robust policy. Through extensive simulated and real experiments, we show significantly better categorical generalization compared to prior approaches in imitation learning settings. Open-sourced code and videos can be found on our website: https://xingyu-lin.github.io/spawnnet.
