Diffuse-XL

What if context compression is a diffusion noise function? Proposal + honest results from untrained-model experiments [R]

<!-- SC_OFF --><div class="md"><p>I'm proposing a way to handle massive context longer than a model's context window by treating semantic compression as the noise function of a diffusion-like process. Instead of denoising masked tokens into coherent text (like DiffusionGemma or Nemotron-Diffusion do for generation), the model reads the source document in multiple passes at decreasing compression levels, heavy summary first, verbatim last all the while it iteratively refines an &quot;integration

DiffusionGemma: 4x faster text generation

Safe Autoregressive Image Generation with Iterative Self-Improving Codebooks

Unlike diffusion-based models that operate in continuous latent spaces, autoregressive unified multimodal models produce images by sequentially predicting discretized visual tokens. These tokens are derived from a codebook that maps embeddings to quantized visual patterns. The language-like architecture enables unified multimodal models to effectively capture text conditional information for generation, making them promising for text-to-image tasks. This also raises an interesting question: how safe are the images generated in such an autoregressive way? In this work, we propose iterative self

minimal-diffusion-lm

A tiny, readable diffusion language model.

Understand diffusion in 20 minutes

The forward and reverse process, with runnable code.

Diffusion

Generating data by gradually removing noise, step by step.