SimpleOPD: Simple Tokenizer-Agnostic On-Policy Distillation for Long-Context Reasoning
On-policy distillation (OPD) offers a promising way to transfer reasoning capabilities from stronger teacher models, but applying it to long-context reasoning teachers and short-context students introduces practical challenges, including tokenizer mismatch, teacher-student distribution mismatch, response length explosion, and training instability. In this work, we study this setting by transferring proof-reasoning capabilities from the long-context reasoning model SU-01 to short-context student models. To handle tokenizer differences, we perform OPD in a shared text space and align only tokens
Lineage graph
Paper → model → repo connections mined from source citations (Tier-1 exact match).
Why these links exist
Every edge carries a method, confidence, and the source snippet that justified it — so bad links are debuggable.
- FuzzyOverlapping authors or contributors · 62%bytedance/deer-flow →
“Shared author/contributor keys: wang”
- FuzzyOverlapping authors or contributors · 62%sgl-project/sglang →
“Shared author/contributor keys: luo”
- FuzzyOverlapping authors or contributors · 62%ray-project/ray →
“Shared author/contributor keys: wang”
- FuzzyOverlapping authors or contributors · 62%DietrichGebert/ponytail →
“Shared author/contributor keys: cheng”
- PossiblePossibly related (embedding) · 46%Paper claims RL for reasoning only changes 1-3% of tokens, and they replicate the gains without RL at ~1000x less compute →
- LinkedLinked via arxiv author · 85%Haonan He →
“SimpleOPD: Simple Tokenizer-Agnostic On-Policy Distillation for Long-Context Reasoning”
- LinkedLinked via arxiv author · 85%Haodi Lei →
“SimpleOPD: Simple Tokenizer-Agnostic On-Policy Distillation for Long-Context Reasoning”
- LinkedLinked via arxiv author · 85%Yun Luo →
“SimpleOPD: Simple Tokenizer-Agnostic On-Policy Distillation for Long-Context Reasoning”