Synthetic Persona Pretraining: Alignment from Token Zero
As language-model-based AI is increasingly deployed in autonomous settings, aligning its goals and values with those of humans becomes critical. Today, alignment, and the assistant identity itself, are typically introduced only after pretraining, once behavioral priors are already established. This can make values a thin overlay, rather than deeply rooted, and facilitate subsequent misalignment. Pursuing a different paradigm, we introduce Synthetic Persona Pretraining (SPP), which installs the desired assistant persona from token zero in pretraining. First, we annotate pretraining documents wi
Lineage graph
Paper → model → repo connections mined from source citations (Tier-1 exact match).
Why these links exist
Every edge carries a method, confidence, and the source snippet that justified it — so bad links are debuggable.
- PossiblePossibly related (embedding) · 50%How Preply combines AI and human tutors to personalize learning →
- FuzzyOverlapping authors or contributors · 62%open-webui/open-webui →
“Shared author/contributor keys: nguyen”
- LinkedLinked via arxiv author · 85%Julian Minder →
“Synthetic Persona Pretraining: Alignment from Token Zero”
- LinkedLinked via arxiv author · 85%Viktor Moskvoretskii →
“Synthetic Persona Pretraining: Alignment from Token Zero”
- LinkedLinked via arxiv author · 85%Raghav Singhal →
“Synthetic Persona Pretraining: Alignment from Token Zero”
- LinkedLinked via arxiv author · 85%Difan Jiao →
“Synthetic Persona Pretraining: Alignment from Token Zero”
- LinkedLinked via arxiv author · 85%Andy Arditi →
“Synthetic Persona Pretraining: Alignment from Token Zero”
- LinkedLinked via arxiv author · 85%Shaobo Cui →
“Synthetic Persona Pretraining: Alignment from Token Zero”