セルフノート:メモリ依存操作タスクのためのスクラッチパッド拡張VLA
Notes-to-Self: Scratchpad Augmented VLAs for Memory Dependent Manipulation Tasks
言語スクラッチパッドをVLAに導入し、空間・時間メモリを付与することで、メモリ依存の長期的操作タスクの汎化性能を向上させた研究。
詳しい要約
1. どんなもの?
2. 先行研究と比べてどこがすごい?
3. 技術・手法の肝は?
4. どうやって有効だと検証した?
5. 議論はある?
6. 次に読むべき論文は?
※ AIが要旨から生成した要約です。正確性は原文をご確認ください。
著者: Sanjay Haresh, Daniel Dijkman, Apratim Bhattacharyya, Roland Memisevic
分類: cs.RO
原文アブストラクト
Many dexterous manipulation tasks are non-markovian in nature, yet little attention has been paid to this fact in the recent upsurge of the vision-language-action (VLA) paradigm. Although they are successful in bringing internet-scale semantic understanding to robotics, existing VLAs are primarily "stateless" and struggle with memory-dependent long horizon tasks. In this work, we explore a way to impart both spatial and temporal memory to a VLA by incorporating a language scratchpad. The scratchpad makes it possible to memorize task-specific information, such as object positions, and it allows the model to keep track of a plan and progress towards subgoals within that plan. We evaluate this approach on a split of memory-dependent tasks from the ClevrSkills environment, on MemoryBench, as well as on a challenging real-world pick-and-place task. We show that incorporating a language scratchpad significantly improves generalization on these tasks for both non-recurrent and recurrent models.
