Multimodal Reward Hacking in Reinforcement Learning
Reinforcement learning (RL) is increasingly used to align multimodal large language models (MLLMs), but higher rewards do not always imply better task performance. This risk is amplified when visual evidence is evaluated by text-only or weakly grounded rewards. We study reward hacking in MLLM RL across safety VQA, chart VQA, and stress-test settings, varying reward design, data ambiguity, model scale (2B-32B), and RL algorithm (GRPO, RLOO, DAPO). We introduce Newly Rewarded Failure Rate (NRFR), which measures failures among samples whose proxy reward improves over the SFT baseline. Outcome-onl
Lineage graph
Paper → model → repo connections mined from source citations (Tier-1 exact match).
Why these links exist
Every edge carries a method, confidence, and the source snippet that justified it — so bad links are debuggable.
- PossiblePossibly related (embedding) · 57%A debugger for RL reward functions that detects reward hacking during training [P] →
- PossiblePossibly related (embedding) · 54%hscspring/rl-llm-nlp →
- LinkedLinked via arxiv author · 85%Jiayu Yao →
“Multimodal Reward Hacking in Reinforcement Learning”
- LinkedLinked via arxiv author · 85%Yiwei Wang →
“Multimodal Reward Hacking in Reinforcement Learning”
- LinkedLinked via arxiv author · 85%Anmeng Zhang →
“Multimodal Reward Hacking in Reinforcement Learning”
- LinkedLinked via arxiv author · 85%Zhe Sun →
“Multimodal Reward Hacking in Reinforcement Learning”
- LinkedLinked via arxiv author · 85%Songsong Wang →
“Multimodal Reward Hacking in Reinforcement Learning”
- LinkedLinked via arxiv author · 85%Lingrui Mei →
“Multimodal Reward Hacking in Reinforcement Learning”