Consistency Models

The Problem

Even with advanced samplers like DDIM or DPM-Solver, diffusion models still required 10 to 50 sequential passes through a massive neural network to generate an image. This iterative process fundamentally prevented diffusion models from matching the real-time speed of GANs. Researchers tried "distilling" multi-step models into fewer steps, but the quality degraded rapidly.

The Idea

The authors (including the creator of Score-Based SDEs) proposed Consistency Models. Instead of learning to remove a tiny bit of noise at each step (like DDPM), a consistency model learns to map any noisy point directly to the final, clean image. The core concept is "self-consistency": if $x_t$ and $x_{t+1}$ belong to the exact same diffusion trajectory, the model should output the exact same final image for both of them. By enforcing this consistency during training, the model learns to jump from pure noise to the final image in a single step.

How It Works

The model learns a function $f(x, t)$ with the property that $f(x_t, t) = x_0$ for all $t$ on a given trajectory.\n\nTraining (Distillation): You take a pre-trained, slow diffusion model. You sample a point $x_{t+1}$, use the slow model to take one tiny denoising step to get $x_t$. You then feed both $x_{t+1}$ and $x_t$ into the Consistency Model. The loss function penalizes the model if $f(x_{t+1}) \neq f(x_t)$. Because $f(x_0) = x_0$ by definition, this chain of consistencies forces the model to learn the direct mapping to the origin.\n\nGeneration: To generate an image, you sample pure noise $x_T$ and simply evaluate $f(x_T, T)$. The image is generated in exactly one network pass. If you want higher quality, you can optionally take a few alternating steps of adding noise and applying the consistency function.

Why It Mattered

Consistency Models finally bridged the speed gap between diffusion models and GANs, allowing high-quality, continuous-time diffusion models to run in real-time (e.g., 30 frames per second). It represented the most mathematically sound approach to distillation.

What Came After

Consistency Models evolved into Latent Consistency Models (LCMs), which were applied to Stable Diffusion. LCMs became wildly popular in the open-source community, allowing users to generate high-quality SDXL images in 2-4 steps. It enabled real-time interactive drawing apps and live-camera filters powered by diffusion.