Paper breakdown

U-Net

Introduced U-Net, an elegant, symmetric architecture for image segmentation that remains the foundational backbone for modern diffusion models.

Paper: U-Net: Convolutional Networks for Biomedical Image Segmentation

Authors: Olaf Ronneberger, Philipp Fischer, Thomas Brox · 2015

Read the paper

The U-Net architecture features a contracting path (encoder) and an expansive path (decoder) connected by lateral skip connections.

The Problem

In many computer vision tasks, particularly in the medical field, it's not enough to say 'there is a dog in this image'. You need to assign a class label to every single pixel in the image (semantic segmentation). Standard CNNs progressively shrink the spatial resolution of the image to extract high-level semantic meaning, which destroys the fine spatial details needed to draw exact pixel boundaries.

The Idea

The authors created a symmetric 'U-shaped' architecture. An encoder shrinks the image to learn the 'what' (context), and a decoder expands it back up to learn the 'where' (localization). Crucially, they added horizontal skip connections that bridge the two sides, copying high-resolution details directly from the encoder to the decoder.

How It Works

Contracting Path (Left side): A standard CNN that repeatedly applies convolutions and max pooling. Spatial resolution drops, but the number of feature channels (semantic understanding) increases.
Expansive Path (Right side): Uses transposed convolutions to upsample the image back to its original resolution.
Skip Connections: At every stage of the expansion, the feature map from the corresponding stage of the contracting path is concatenated to the current map. This gives the decoder access to the exact spatial details that were lost during pooling, allowing it to draw highly precise boundaries.

Why It Mattered

U-Net achieved state-of-the-art segmentation with very little training data (crucial for medical imaging, where labeled data is scarce). It proved that skip connections across an encoder-decoder structure were the perfect solution for tasks requiring both high-level semantic context and low-level spatial precision.

What Came After

U-Net became the undisputed standard for biomedical image segmentation. However, its most massive impact was yet to come. Because U-Net is brilliant at taking a noisy image and predicting a cleaned-up image of the exact same size, it was chosen as the architectural backbone for modern Diffusion Models (like Stable Diffusion and Midjourney).