What Does a Discrete Diffusion Model Learn?
What does a discrete diffusion model learn: a denoiser, a score ratio, or a bridge plug-in predictor? At the level of jump rates, these are one object in different coordinates, and reading a neural network in the wrong coordinate changes the process being trained and sampled. Starting with a rigorous derivation of the continuous-time Markov chain (CTMC) ELBO for any noising process, boundary terms included, we prove the \emph{Oracle Distance} theorem: the negative ELBO is exactly equal to the data entropy plus the path KL from the oracle reverse process to the learned one, not merely a bound.
Lineage graph
Paper → model → repo connections mined from source citations (Tier-1 exact match).
Why these links exist
Every edge carries a method, confidence, and the source snippet that justified it — so bad links are debuggable.
- PossiblePossibly related (embedding) · 52%Understand diffusion in 20 minutes →
- PossiblePossibly related (embedding) · 49%minimal-diffusion-lm →
- PossiblePossibly related (embedding) · 48%Diffusion →
- PossiblePossibly related (embedding) · 47%Scaling Properties of Continuous Diffusion Spoken Language Models - Apple Machine Learning Research →
- LinkedLinked via arxiv author · 85%Rodrigo Casado Noguerales →
“What Does a Discrete Diffusion Model Learn?”
- LinkedLinked via arxiv author · 85%Bernhard Schölkopf →
“What Does a Discrete Diffusion Model Learn?”
- LinkedLinked via arxiv author · 85%Thomas Hofmann →
“What Does a Discrete Diffusion Model Learn?”
- LinkedLinked via arxiv author · 85%Aran Raoufi →
“What Does a Discrete Diffusion Model Learn?”