Convolution / CNN

What you are looking at

Three grids, left to right. On the left is the input image, shown as a 6x6 grid of pixel intensities. In the middle are the nine weights of the kernel — the filter itself, printed rather than hidden. On the right is the 4x4 feature map the two of them produce.

The solid outline on the input is the window the kernel currently sits over. The dashed outline on the feature map is the single cell that window produces. Move Kernel Position and the two move together.

The Dot Product

The calculation is an element-wise multiplication followed by a sum.

If your kernel is 3x3, it looks at a 3x3 patch of the image. It multiplies the top-left pixel of the patch by the top-left weight of the kernel, the top-middle by the top-middle, and so on. Then it adds all nine results together.

That single sum becomes one pixel in the output feature map. It appears under the kernel grid as a running Σ, and again as the Dot Product Sum readout.

A magnifying glass for patterns

Think of the kernel as a template or a stencil. When the pattern in the image patch matches the pattern in the kernel weights, the multiplication produces large positive numbers, and their sum is high.

When the image patch doesn't match the kernel at all, the positive and negative results cancel out, leaving a sum near zero. The kernel is literally scanning the image asking, "Is my specific pattern right here?"

Weight Sharing

Notice that as you slide the kernel, the weights inside it never change. The exact same filter is applied to the top-left corner as to the bottom-right corner.

This is the core insight of CNNs, called weight sharing. A vertical edge looks like a vertical edge no matter where it appears in the image. By reusing the same small set of weights across the entire image, the network drastically reduces the number of parameters it needs to learn, while gaining translation invariance (the ability to recognize a feature regardless of its position).

Different Kernels, Different Features

Change Kernel Type and the output map changes entirely. The middle grid changes with it, so you can read the weights that caused the change rather than infer them.

An edge detection kernel uses positive weights on one side and negative weights on the other — visible in the middle grid as one column or row of one hue against the opposite hue. It produces a high score only when it sits exactly on a boundary where dark pixels meet light pixels. On the Vertical Edge image the vertical detector reads 3 at the boundary and 0 everywhere else.

A blur kernel is nine weights of 1/9, so it returns the mean of the patch. That is what keeps it a blur: nine weights of 1 would return the sum, and a 0-to-1 image would come out with values up to 9. Watch the feature map ramp 0 to 0.33 to 0.67 to 1 across the boundary — the same range as the input, with the step smoothed into a slope.

A sharpen kernel weights the centre pixel at 5 and subtracts its four neighbours. Its weights also sum to 1, but it overshoots: on the Vertical Edge image the feature map dips to −1 just before the boundary and rises to 2 just after, which is the halo you see around an over-sharpened photograph.

In a real neural network, you don't hand-code these weights. The network starts with random kernels and, through backpropagation, learns the specific numbers that extract the most useful features for its task — perhaps edges in the first layer, textures in the second, and complex shapes in deeper layers.

The Shrinking Image

If you apply a 3x3 kernel to a 6x6 image without adding any artificial borders, the kernel can only fit in 4 horizontal positions and 4 vertical positions before it falls off the edge.

As a result, your 6x6 image produces a 4x4 feature map.

To prevent the image from shrinking at every layer, real networks often use padding — adding a border of zeros around the original image so the kernel has room to slide further, keeping the output map the exact same size as the input.

The failure of the missing feature

Set Kernel Type to Horizontal Edge and leave Image on Vertical Edge. Or set Image to Horizontal Edge and Kernel Type back to Vertical Edge. Either way, all 16 cells of the feature map read exactly 0 and Cells Firing drops to zero.

This isn't a bug; it is the correct mathematical result. The Horizontal Edge kernel subtracts its top row of pixels from its bottom row, so it reads down the image — and the Vertical Edge image has six identical rows, so there is nothing to subtract. The nine products cancel at all 16 positions. The dot product is telling you the specific feature this kernel cares about does not exist in this input.

Note that the matching name is the one that works: the Vertical Edge kernel fires on the Vertical Edge image. It is the mismatched pairs that go silent.

In a real network, this is why a single layer uses hundreds of different kernels simultaneously — if one filter finds nothing, another might find exactly what it's looking for. Switch Image to Corner and both edge detectors fire, on different cells.