A Learning-Rate-Gated Failure of GRPO in a Small Language and Vision-Language Model Web Agent: A Controlled Null and Its Mechanism
Reinforcement learning with verifiable rewards, and Group Relative Policy Optimization (GRPO) in particular, is now run routinely on a supervised checkpoint in the hope of producing a stronger agent. We ask whether it adds skill to a small language and vision-language model web agent at the 4B to 8B scale, or whether it mostly reshapes behavior the supervised model already has. Across a control grid of 18 runs that varies learning rate, KL weight, seed, initialization, and clipping, no configuration credibly improves the success rate of a strong supervised baseline on tasks the agent has large
Lineage graph
Paper → model → repo connections mined from source citations (Tier-1 exact match).
Why these links exist
Every edge carries a method, confidence, and the source snippet that justified it — so bad links are debuggable.
- PossiblePossibly related (embedding) · 54%hscspring/rl-llm-nlp →
- PossiblePossibly related (embedding) · 53%AgileRL/AgileRL →
- PossiblePossibly related (embedding) · 52%Weblica: Scalable and Reproducible Training Environments for Visual Web Agents - Apple Machine Learning Research →
- FuzzySimilar title/name (fuzzy) · 59%AgentCore-8B →
“Fuzzy title match (0.73): “A Learning-Rate-Gated Failure of GRPO in a Small Language an” ≈ “AgentCore-8B””
- FuzzySimilar title/name (fuzzy) · 59%VioletVision-3B →
“Fuzzy title match (0.73): “A Learning-Rate-Gated Failure of GRPO in a Small Language an” ≈ “VioletVision-3B””
- LinkedLinked via arxiv author · 85%Chengguang Gan →
“A Learning-Rate-Gated Failure of GRPO in a Small Language and Vision-Language Model Web Agent: A Controlled Null and Its”
- LinkedLinked via arxiv author · 85%Zhixi Cai →
“A Learning-Rate-Gated Failure of GRPO in a Small Language and Vision-Language Model Web Agent: A Controlled Null and Its”
- LinkedLinked via arxiv author · 85%Yunhao Liang →
“A Learning-Rate-Gated Failure of GRPO in a Small Language and Vision-Language Model Web Agent: A Controlled Null and Its”