Offline Deep Q* Estimation with Diffusion Models
In offline RL, estimating the optimal action-value function $Q^*$ can be formulated as solving the optimal Bellman equation based solely on offline observations. A fundamental challenge is that the reward function and transition kernel are unknown, so the optimal Bellman operator is not directly observable from data. To address this issue, we propose a novel framework that decouples operator estimation from value function learning. In this approach, we first formulate conditional diffusion models to estimate the reward law and transition kernel, which induces a data-driven approximation of the
Lineage graph
Paper → model → repo connections mined from source citations (Tier-1 exact match).
Why these links exist
Every edge carries a method, confidence, and the source snippet that justified it — so bad links are debuggable.
- FuzzySimilar title/name (fuzzy) · 59%CompVis/stable-diffusion-v1-4 →
“Fuzzy title match (0.73): “Offline Deep Q* Estimation with Diffusion Models” ≈ “CompVis/stable-diffusion-v1-4””
- PossiblePossibly related (embedding) · 52%RL without TD learning →
- PossiblePossibly related (embedding) · 46%[2607.07508] Single-Rollout Asynchronous Optimization for Agentic Reinforcement Learning →
- PossiblePossibly related (embedding) · 46%Adaptive primal–dual Q-learning for electric vehicle route optimization on real-world charging networks - Nature →
- LinkedLinked via arxiv author · 85%Xiaohong Chen →
“Offline Deep Q* Estimation with Diffusion Models”
- LinkedLinked via arxiv author · 85%Yuling Jiao →
“Offline Deep Q* Estimation with Diffusion Models”
- LinkedLinked via arxiv author · 85%Lican Kang →
“Offline Deep Q* Estimation with Diffusion Models”
- LinkedLinked via arxiv author · 85%Jerry Zhijian Yang →
“Offline Deep Q* Estimation with Diffusion Models”