次トークン予測器における一見無用な特徴量の出現の理解
Understanding the Emergence of Seemingly Useless Features in Next-Token Predictors
Transformerが次トークン予測に不要に見える抽象的特徴を学習する理由を勾配信号の観点から分析し、その影響を推定する手法を提案した論文。
詳しい要約
1. どんなもの?
2. 先行研究と比べてどこがすごい?
3. 技術・手法の肝は?
4. どうやって有効だと検証した?
5. 議論はある?
6. 次に読むべき論文は?
※ AIが要旨から生成した要約です。正確性は原文をご確認ください。
著者: Mark Rofin, Jalal Naghiyev, Michael Hahn
分類: cs.LG, cs.CL
原文アブストラクト
Trained Transformers have been shown to compute abstract features that appear redundant for predicting the immediate next token. We identify which components of the gradient signal from the next-token prediction objective give rise to this phenomenon, and we propose a method to estimate the influence of those components on the emergence of specific features. After validating our approach on toy tasks, we use it to interpret the origins of the world model in OthelloGPT and syntactic features in a small language model. Finally, we apply our framework to a pretrained LLM, showing that features with extremely high or low influence on future tokens tend to be related to formal reasoning domains such as code. Overall, our work takes a step toward understanding hidden features of Transformers through the lens of their development during training.
