How to Build a Diffusion Language Model
An introduction to diffusion language models and the research advances that underlie today's diffusion LLMs. We describe the building blocks of recent open-source models, starting from simple masking diffusion, and including techniques for iterative refinement, post-training, and variable-length generation. Material is adapted from workshop talks and lectures at ICLR 2026 and MLSS 2026.
-
How to Build a Diffusion Language Model
An introduction to diffusion language models and the research advances that underlie today's diffusion LLMs. We describe the building blocks of recent open-source models, starting from simple masking diffusion, and including techniques for iterative refinement, post-training, and variable-length generation. Material is adapted from workshop talks and lectures at ICLR 2026 and MLSS 2026.
-
Set Diffusion: Interpolating Token Orderings Between Autoregression and Diffusion for Fast and Flexible Decoding
SetDLM generalizes block diffusion by decoding flexible-position, flexible-size token sets instead of fixed blocks, enabling cacheable, infilling-capable, higher-throughput generation. Published at ICML 2026.
-
Discrete Diffusion: A New Frontier For Language Modeling
We introduce diffusion language models and explain their main advantages over standard autoregerssive models using existing large diffusion models as examples. We also highlight diffusion for language as an emerging research direction in generative AI. Adapted from an invited workshop talk at ICML 2025.
-
Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models
BD3-LMs interpolate between autoregressive and diffusion language models by generating blocks of tokens, supporting variable-length, KV-cached, higher-quality generation. ICLR 2025 (Oral).
-
Remasking Discrete Diffusion Models with Inference-Time Scaling
ReMDM introduces a remasking sampler that lets masked diffusion models revise previously generated tokens, unlocking inference-time compute scaling for better sample quality. Published at NeurIPS 2025.