Diffusion Models

What you are looking at

A simulation of a Diffusion Model in action, using a 2D particle scatter plot. The main visualization shows a structured geometric shape (a spiral) as it transitions between pure structure and pure noise.

Modern image generators like Midjourney or Stable Diffusion are based on this core principle: if you can slowly destroy a structure by adding noise, and train a neural network to predict the noise that was added, you can reverse the process to generate new data from pure static. (While real image generators diffuse millions of pixels, the math is identical whether you are diffusing pixels in an image or $(x,y)$ coordinates in a 2D plot).

The Forward Process

When you drag the timeline slider forward, you are simulating the Forward Process ($q(x_t|x_0)$).

In this phase, we take our clean training data and iteratively add small amounts of Gaussian noise. We do this over many steps (here, 50 steps; real models use 1,000+). By the final step, all original structure is completely obliterated, leaving nothing but an isotropic Gaussian cloud (a meaningless circle of static).

This process has no learnable parameters. It's just a fixed mathematical operation used to create training examples for the network.

The Beta Schedule

Notice the variance schedule curve tracking $\beta$ (beta). The amount of noise we add isn't constant at every step.

Diffusion models typically use a schedule where $\beta$ increases over time. Early on (near step 1), we add very tiny amounts of noise, keeping the original shape mostly intact. As we get closer to step 50, we start throwing larger and larger amounts of noise at it. This ensures a smooth transition from structure to static, giving the neural network examples across the entire spectrum of signal-to-noise ratios.

The Reverse Process

When you drag the timeline backward, you are simulating the Reverse Process ($p(x_{t-1}|x_t)$).

This is where the magic happens. A neural network (usually a U-Net or Transformer) is given a noisy point cloud and asked: "What noise was added to get here?" Once it predicts the noise (the vectors shown in the secondary view), we subtract a small portion of it to step slightly backwards in time, recovering a tiny bit of structure.

By starting at pure static and iteratively applying the neural network, we slowly carve the spiral out of the noise.

Why take so many steps?

You might wonder why we don't just ask the network to generate the final shape in one single step.

The relationship between pure, unstructured noise and a highly structured distribution is incredibly complex and non-linear. If we ask the network to jump from step 50 to step 0 in one shot, it fails completely, producing a blurry mess. However, asking the network to remove just a tiny fraction of noise to get from step 50 to step 49 is a much simpler, tractable problem. By breaking the impossible task into 50 easy tasks, generative AI becomes possible.