CLIPよ、お前もか?未見環境における一般的な物体誤りへの対処
ET tu, CLIP? Addressing Common Object Errors for Unseen Environments
ALFREDタスクにおいて、事前学習済みCLIPエンコーダを補助的な物体検出目的で追加モジュールとして用いることで、未見環境でのモデル汎化性能を向上させる手法を提案した。
著者: Ye Won Byun, Cathy Jiao, Shahriar Noroozizadeh, Jimin Sun, Rosa Vitiello
分類: cs.CV, cs.AI, cs.CL, cs.LG, cs.RO
原文アブストラクト
We introduce a simple method that employs pre-trained CLIP encoders to enhance model generalization in the ALFRED task. In contrast to previous literature where CLIP replaces the visual encoder, we suggest using CLIP as an additional module through an auxiliary object detection objective. We validate our method on the recently proposed Episodic Transformer architecture and demonstrate that incorporating CLIP improves task performance on the unseen validation set. Additionally, our analysis results support that CLIP especially helps with leveraging object descriptions, detecting small objects, and interpreting rare words.
