MemDefrag: Latent Memory Defragmentation for Large Language Models
Latent memory, which stores past knowledge fragments as per-layer hidden states, has emerged as a promising paradigm (e.g., MemoryLLM and M+) for long-term memory in large language models (LLMs). However, the paradigm suffers from significant performance degradation during memory updates, due to positional encoding misalignment and the absence of any tracing mechanism to distinguish target memory fragments from irrelevant ones. To discover such a tracing mechanism, we probe the layer-wise attention density over stored memory fragments, and find that a small set of middle transformer layers con
Lineage graph
Paper → model → repo connections mined from source citations (Tier-1 exact match).
Why these links exist
- Linked via arxiv authorRuiyi Yan →
MemDefrag: Latent Memory Defragmentation for Large Language Models
- Linked via arxiv authorZhuoyuan Mao →
MemDefrag: Latent Memory Defragmentation for Large Language Models
- Linked via arxiv authorYiwen Guo →
MemDefrag: Latent Memory Defragmentation for Large Language Models