When Does Muon Help Agentic Reinforcement Learning?
Muon is competitive with AdamW in large-scale pre-training, but its value for reinforcement-learning (RL) post-training remains unclear. We study vanilla Muon in sparse-reward agentic RL through matched single-seed comparisons with AdamW on ALFWorld using Qwen2.5-0.5B-Instruct. Under Group-in-Group Policy Optimization (GiGPO), applying Muon only to hidden weight matrices raises final-window validation success from 0.290 to 0.546 (+88%); high-rate AdamW controls retain no post-update success. The effect depends on the advantage estimator and learning rate. At 3e-5, Muon improves GRPO from 0.161
Lineage graph
Paper → model → repo connections mined from source citations (Tier-1 exact match).
Why these links exist
Every edge carries a method, confidence, and the source snippet that justified it — so bad links are debuggable.
- PossiblePossibly related (embedding) · 46%[2607.07508] Single-Rollout Asynchronous Optimization for Agentic Reinforcement Learning →
- LinkedLinked via arxiv author · 85%Kai Ruan →
“When Does Muon Help Agentic Reinforcement Learning?”
- LinkedLinked via arxiv author · 85%Jinghao Lin →
“When Does Muon Help Agentic Reinforcement Learning?”
- LinkedLinked via arxiv author · 85%Zihe Huang →
“When Does Muon Help Agentic Reinforcement Learning?”
- LinkedLinked via arxiv author · 85%Ziqi Zhou →
“When Does Muon Help Agentic Reinforcement Learning?”
- LinkedLinked via arxiv author · 85%Qianshan Wei →
“When Does Muon Help Agentic Reinforcement Learning?”
- LinkedLinked via arxiv author · 85%Zixuan Wang →
“When Does Muon Help Agentic Reinforcement Learning?”
- LinkedLinked via arxiv author · 85%Mingchao Sun →
“When Does Muon Help Agentic Reinforcement Learning?”