Measuring Task-Agnostic Training Data Influence Across Language Model Pretraining
Measuring training data influence consistently across language model pretraining is challenging. It is difficult to select downstream tasks or validation sets representative of a model's general capabilities, and reliance on task performance at intermediate checkpoints complicates comparisons across training. We propose a measure of training data influence that does not require selecting a downstream task or validation set as the attribution target. Specifically, we define an example's influence by how much its gradient update reduces the squared distance to the final parameters of a given pre
Lineage graph
Paper → model → repo connections mined from source citations (Tier-1 exact match).
Why these links exist
Every edge carries a method, confidence, and the source snippet that justified it — so bad links are debuggable.
- FuzzyOverlapping authors or contributors · 62%modular/modular →
“Shared author/contributor keys: liu”
- LinkedLinked via arxiv author · 85%Yuto Nishida →
“Measuring Task-Agnostic Training Data Influence Across Language Model Pretraining”
- LinkedLinked via arxiv author · 85%Hirokazu Kiyomaru →
“Measuring Task-Agnostic Training Data Influence Across Language Model Pretraining”
- LinkedLinked via arxiv author · 85%Yusuke Oda →
“Measuring Task-Agnostic Training Data Influence Across Language Model Pretraining”
- LinkedLinked via arxiv author · 85%Takashi Kodama →
“Measuring Task-Agnostic Training Data Influence Across Language Model Pretraining”
- LinkedLinked via arxiv author · 85%Chaoran Liu →
“Measuring Task-Agnostic Training Data Influence Across Language Model Pretraining”
- LinkedLinked via arxiv author · 85%Daisuke Kawahara →
“Measuring Task-Agnostic Training Data Influence Across Language Model Pretraining”
- LinkedLinked via arxiv author · 85%Yusuke Miyao →
“Measuring Task-Agnostic Training Data Influence Across Language Model Pretraining”