Towards Mechanistically Understanding Why Memorized Knowledge Fails to Generalize in Large Language Model Finetuning
Fine-tuning LLMs to inject new knowledge faces a critical challenge: LLMs can quickly memorize new facts, yet fail to use them for downstream reasoning tasks. We formalize this failure as the \textit{\textbf{Knowing--Using Gap}}, characterized by an accuracy gap and a temporal lag between memorization and generalization. To understand this phenomenon, we fine-tune LLMs with unseen knowledge and monitor the spatial permeation dynamics of the knowledge internally using a novel intervention technique called self-patching. Self-patching identifies activation locations where relocating representati
Lineage graph
Paper → model → repo connections mined from source citations (Tier-1 exact match).
Why these links exist
Every edge carries a method, confidence, and the source snippet that justified it — so bad links are debuggable.
- PossiblePossibly related (embedding) · 59%douglasjordan2/c0 →
- PossiblePossibly related (embedding) · 58%llmsresearch/llm-flashcards →
- PossiblePossibly related (embedding) · 57%chrisliu298/awesome-llm-unlearning →
- PossiblePossibly related (embedding) · 53%amitshekhariitbhu/llm-internals →
- PossiblePossibly related (embedding) · 53%MemTensor/MemOS →
- LinkedLinked via arxiv author · 85%Lu Dai →
“Towards Mechanistically Understanding Why Memorized Knowledge Fails to Generalize in Large Language Model Finetuning”
- LinkedLinked via arxiv author · 85%Ziyang Rao →
“Towards Mechanistically Understanding Why Memorized Knowledge Fails to Generalize in Large Language Model Finetuning”
- LinkedLinked via arxiv author · 85%Yili Wang →
“Towards Mechanistically Understanding Why Memorized Knowledge Fails to Generalize in Large Language Model Finetuning”