A Hippocampus for Linear Attention: An Exact Memory for What the Recurrent State Forgets
Linear-attention and state-space language models compress the prefix into a fixed-size recurrent state, yielding O(1) memory at the cost of a lossy exact memory: when many key--value associations compete, earlier facts are overwritten and needle recall degrades. Inspired by Complementary Learning Systems, we give linear attention a hippocampal complement. HOLA (Hippocampal Linear Attention) keeps the usual delta-rule state as a compressive memory and adds a bounded exact KV cache, forming a semiparametric test-time memory: the state models linearly compressible structure, while the cache store
Lineage graph
Paper → model → repo connections mined from source citations (Tier-1 exact match).
Why these links exist
Every edge carries a method, confidence, and the source snippet that justified it — so bad links are debuggable.
- PossiblePossibly related (embedding) · 50%Matrix Orthogonalization Improves Memory in Recurrent Models →
- PossiblePossibly related (embedding) · 47%Breakthrough in long-context efficiency announced →
- PossiblePossibly related (embedding) · 49%Hierarchos: Preliminary Findings From a 232M Recurrent Memory-Augmented Assistant Model [P] →
- LinkedLinked via arxiv author · 85%Wanyun Cui →
“A Hippocampus for Linear Attention: An Exact Memory for What the Recurrent State Forgets”
- PossiblePossibly related (embedding) · 52%cdeust/Cortex →
- PossiblePossibly related (embedding) · 48%ActiveMemory/ctx →
- PossiblePossibly related (embedding) · 74%A Hippocampus for Linear Attention: An Exact Memory for What the Recurrent State Forgets →
- PossiblePossibly related (embedding) · 55%How can we solve long-range recall in linear attention? [D] →