Is One Layer Enough? Training A Single Transformer Layer Can Match Full-Parameter RL Training
Reinforcement learning (RL) has become a central component of post-training large language models (LLMs), yet little is understood about how RL adaptation is distributed across transformer layers. Existing approaches typically update all model parameters uniformly, implicitly assuming that every layer contributes similarly to the gains obtained during RL post-training. In this work, we challenge this assumption through a systematic layer-wise study of RL training. Surprisingly, we find that training a single transformer layer can recover most of the gains achieved by full-parameter RL training
Lineage graph
Paper → model → repo connections mined from source citations (Tier-1 exact match).
Why these links exist
Every edge carries a method, confidence, and the source snippet that justified it — so bad links are debuggable.
- LinkedLinked via unknownTransformer →
- LinkedLinked via unknownRL without TD learning →
- LinkedLinked via unknownRLHF →
- LinkedLinked via unknownIs One Layer Enough? A Single Transformer Layer Matches Full-Parameter RL Train →
- PossiblePossibly related (embedding) · 52%teilomillet/retrain →
- LinkedLinked via arxiv author · 85%Zijian Zhang →
“Is One Layer Enough? Training A Single Transformer Layer Can Match Full-Parameter RL Training”
- LinkedLinked via arxiv author · 85%Rizhen Hu →
“Is One Layer Enough? Training A Single Transformer Layer Can Match Full-Parameter RL Training”
- LinkedLinked via arxiv author · 85%Athanasios Glentis →
“Is One Layer Enough? Training A Single Transformer Layer Can Match Full-Parameter RL Training”