Generalization in offline RL: The structure is more important than the amount of pessimism
While pessimism counteracts overestimation bias in offline reinforcement learning (RL), being overly conservative has been associated with hindering certain forms of generalization. However, in this paper we demonstrate that being overly pessimistic does not inherently prevent optimal generalization in contextual MDPs (CMDPs). Instead, we argue successful generalization depends not on the amount of pessimism, but whether the pessimistic structure respects the underlying symmetries of the optimal solution. We prove that a mildly pessimistic, non-symmetric value function can generalize worse tha
Lineage graph
Paper → model → repo connections mined from source citations (Tier-1 exact match).
Why these links exist
Every edge carries a method, confidence, and the source snippet that justified it — so bad links are debuggable.
- PossiblePossibly related (embedding) · 53%rllm-org/rllm →
- PossiblePossibly related (embedding) · 52%RLHF →
- PossiblePossibly related (embedding) · 46%RL without TD learning →
- LinkedLinked via arxiv author · 85%Max Weltevrede →
“Generalization in offline RL: The structure is more important than the amount of pessimism”
- LinkedLinked via arxiv author · 85%Matthijs T. J. Spaan →
“Generalization in offline RL: The structure is more important than the amount of pessimism”
- LinkedLinked via arxiv author · 85%Wendelin Böhmer →
“Generalization in offline RL: The structure is more important than the amount of pessimism”
- PossiblePossibly related (embedding) · 54%On Adversarial RL [R] →
- PossiblePossibly related (embedding) · 48%The Little Book of Reinforcement Learning →