Weak-to-Strong Generalization via Direct On-Policy Distillation
Reinforcement learning with verifiable rewards (RLVR) is a powerful recipe for improving language-model reasoning, but it is expensive to repeat on every new strong model because the target model must generate many rollouts during training. As models scale, post-training itself becomes a bottleneck. We study a weak-to-strong alternative: run RL on a smaller model where rollouts are cheaper, then reuse what that RL run learned to improve a stronger target model. Directly distilling the post-RL weak teacher is not enough, because the teacher's final policy mixes useful RL gains with the limitati
Lineage graph
Paper → model → repo connections mined from source citations (Tier-1 exact match).
Why these links exist
- Linked via arxiv authorShiyuan Feng →
Weak-to-Strong Generalization via Direct On-Policy Distillation
- Linked via arxiv authorHuan-ang Gao →
Weak-to-Strong Generalization via Direct On-Policy Distillation
- Linked via arxiv authorHaohan Chi →
Weak-to-Strong Generalization via Direct On-Policy Distillation
- Linked via arxiv authorHanlin Wu →
Weak-to-Strong Generalization via Direct On-Policy Distillation
- Linked via arxiv authorZhilong Zhang →
Weak-to-Strong Generalization via Direct On-Policy Distillation
- Linked via arxiv authorZheng Jiang →
Weak-to-Strong Generalization via Direct On-Policy Distillation
- Linked via arxiv authorBingxiang He →
Weak-to-Strong Generalization via Direct On-Policy Distillation
- Linked via arxiv authorWei-Ying Ma →
Weak-to-Strong Generalization via Direct On-Policy Distillation
- Linked via arxiv authorYa-Qin Zhang →
Weak-to-Strong Generalization via Direct On-Policy Distillation
- Linked via arxiv authorHao Zhou →
Weak-to-Strong Generalization via Direct On-Policy Distillation