Persistent Sparse Autoencoders: Learning Feature Timescales in Language Models
Sparse autoencoders (SAEs) decompose language model activations into sparse features, but standard SAEs encode each token independently and do not expose information that persists across a sequence. We introduce Persistent Sparse Autoencoders (Persistent SAEs), which extend standard SAEs by learning a persistence coefficient for each feature, allowing the model to learn which features should persist and for how long. Our experiments show that they retain competitive reconstruction quality while learning a spectrum of feature timescales: fast features behave as locally interpretable detectors,
Lineage graph
Paper → model → repo connections mined from source citations (Tier-1 exact match).
Why these links exist
Every edge carries a method, confidence, and the source snippet that justified it — so bad links are debuggable.
- FuzzyOverlapping authors or contributors · 62%sgl-project/sglang →
“Shared author/contributor keys: luo”
- FuzzySimilar title/name (fuzzy) · 59%aymericdamien/TopDeepLearning →
“Fuzzy title match (0.73): “Persistent Sparse Autoencoders: Learning Feature Timescales ” ≈ “aymericdamien/TopDeepLearning””
- LinkedLinked via arxiv author · 85%Haoyan Luo →
“Persistent Sparse Autoencoders: Learning Feature Timescales in Language Models”
- LinkedLinked via arxiv author · 85%Mateo Espinosa Zarlenga →
“Persistent Sparse Autoencoders: Learning Feature Timescales in Language Models”
- LinkedLinked via arxiv author · 85%Mateja Jamnik →
“Persistent Sparse Autoencoders: Learning Feature Timescales in Language Models”