From Expressivity to Sample Complexity: Narrow Teachers for Transformers via C-RASP
A theoretical understanding of Transformers is crucial to better understand the capacities and limitations of large language models (LLMs). There is much work analyzing the expressivity of attention-based models. By proposing handcrafted weights or using computational complexity arguments, a large amount of past theoretical works have sought to characterize which tasks are and which are not in the hypothesis class of Transformer models. However, little work investigates the learnability of such solutions. In this work, we make progress towards this goal. Inspired by recent loss landscape analy
Lineage graph
Paper → model → repo connections mined from source citations (Tier-1 exact match).
Why these links exist
- Linked via arxiv authorMichael Rizvi-Martel →
From Expressivity to Sample Complexity: Narrow Teachers for Transformers via C-RASP
- Linked via arxiv authorSatwik Bhattamishra →
From Expressivity to Sample Complexity: Narrow Teachers for Transformers via C-RASP
- Linked via arxiv authorGuillaume Rabusseau →
From Expressivity to Sample Complexity: Narrow Teachers for Transformers via C-RASP
- Linked via arxiv authorMichael Hahn →
From Expressivity to Sample Complexity: Narrow Teachers for Transformers via C-RASP