Text Diffusion Models

Text diffusion models generate text by starting with random noise and refining it in multiple passes, unlike standard autoregressive models (which produce one word at a time). This process allows bidirectional reasoning and self-correction — the model can look at all the tokens simultaneously and fix errors as it refines, rather than committing to each word sequentially.

A key advantage is dynamic computation: the model can spend fewer passes on easy prompts and more on hard ones, adapting its effort automatically. This also means fewer memory transfers — generating 256 tokens with 24 refinement passes is 10 times faster than generating them one-by-one if the system is memory-bound. For AI developers, this low latency unlocks faster responses for real-time applications.

These models also support fast in-place editing — you can change part of the generated text without regenerating everything. While most text diffusion today works in discrete space (directly on tokens), combining them with autoregressive models is possible, and reinforcement learning can be applied by changing the training algorithm. For AI engineers, the practical impact is speed and flexibility: easier problems get solved quickly, harder ones get more compute automatically, all while enabling new applications like on-the-fly webpage generation.

Text diffusion models generate text differently than the autoregressive models (models that predict one word at a time) you may know. Instead of producing one token (a word or piece of a word) and feeding it back in, a text diffusion model starts with a full sequence of tokens, like a clean sentence, and gradually adds random noise to corrupt it. Then the model learns to reverse that process, refining the noisy sequence back into a coherent response. Brendan Dillon from Google DeepMind explains that this iterative refinement lets the model generate, say, 256 tokens in 24 passes, which can be ten times faster than autoregressive models when memory is the bottleneck.

To use a text diffusion model step by step, you begin by defining your context or prompt. The model then initializes a fixed window of tokens—often filled with random noise—that matches your desired output length. It denoises this window over multiple steps, correcting the entire sequence at once. This enables bidirectional reasoning (looking at all tokens together) and self-correction, which autoregressive models lack. You can also perform fast in-place editing: like filling in a masked section of an image, you can cut out part of a text and prompt the model to regenerate only that portion using surrounding context.

For example, in a tool like Gemini diffusion, you supply a prompt, and the model determines how many refinement passes to use—simpler prompts get fewer passes, harder ones get more, saving computation. You'll need a compatible setup, such as downloading a Gemma text encoder and the diffusion model itself. Once configured, you select the model, input your text, and let the diffusion process refine it iteratively until you get the final output. The result is text that benefits from creative generation and adaptive speed.

When using text diffusion models (models that generate text by gradually refining a noisy starting sequence), keep these practical points in mind. A key advantage is fewer memory transfers; a diffusion model might refine 256 tokens in 24 passes, while an autoregressive model (generating one token at a time) would require 256 passes. If your hardware is memory-bound, this can make the diffusion model up to ten times faster in latency.

A common pitfall is expecting diffusion models to follow prompts as closely as autoregressive models; they behave more like traditional image diffusion, offering an exploratory feel in latent space (the compressed representation the model works in). This trade-off means you may need to prompt more loosely for creative results.

Another best practice is leveraging the model's ability to self-correct. Autoregressive models cannot fix an early mistake without backtracking — one example showed a model incorrectly stating “36 + 3 is 42” and continuing to build reasoning on that error. Text diffusion models avoid this flaw, allowing bidirectional reasoning and editing. You can also apply reinforcement learning (RL) to these models by changing the training algorithm.

For speed, use the dynamic computation property: the model can spend fewer refinement steps on simple prompts and more on complex ones (e.g., easier Python problems get faster responses). Finally, while you can combine diffusion with autoregressive components, understand the latency-throughput trade-off — slower generation but better handling of reasoning tasks.