Visual explainer

Dropout

How randomly zeroing half the units stops lazy co-adaptation, with train and test sums matched at 4.

One neuron carries 90 percent of the load while three ride along doing nothing.

Left alone, your network gets lazy. One star neuron carries 90 percent while three passengers nap, and every downstream weight trusts the star. New data breaks the star, and the whole act collapses with it.

Flip A Coin Each

Each neuron flips its own coin and sits out half the training steps.

Dropout gives every neuron its own coin with p equal to 0.5. Each step flips fresh coins, so nobody knows which neighbours will show up. Every unit must learn something useful on its own merits.

Half Plays, Sum 4

Two of four units stay live with weights 2 and 2, so the layer sums to exactly 4.

Watch one step with all inputs 1 and all weights 2. Two units stay live, two zero out, and the layer sums 2 plus 0 plus 2 plus 0 to 4. The star cannot carry a team that keeps vanishing.

Everyone Back Halved

All four units return scaled by 0.5, so 1 plus 1 plus 1 plus 1 still sums to 4.

At test time all four units return, each scaled by 0.5 to 1. The sum reads 1 plus 1 plus 1 plus 1, still exactly 4. The next layer sees the volume it trained on, with the full team present.

Gap Shrinks 0.19

The train test gap shrinks from 0.29 to 0.10 while held out error falls 0.31 to 0.18.

Same net, same data, only dropout added. Training error rises 0.02 to 0.08 while held-out error falls 0.31 to 0.18. The gap collapses from 0.29 to 0.10 because redundancy replaced reliance.

Rate By Layer

Drop half the hidden units and a fifth of inputs, and never drop the outputs.

Hidden layers take the standard 0.5 brawl. Raw inputs get a gentle 0.2 since data is precious. Outputs stay at 0.0 because votes must stay whole. Small nets or huge datasets flip you down or off entirely.

Where It Breaks

Dropout left on at test time jitters answers, so call eval mode before you predict.

Forget eval mode and units keep dropping while you predict. One input scores anywhere from 0.62 to 0.88 across five passes. Same weights, pure noise from a forgotten switch, fixed by one model.eval call.

The Quick Version

One star carries 90; three nap.
Each neuron flips its own coin.
Half live sums exactly to 4.
Test scaling keeps the sum 4.
Gap falls from 0.29 to 0.10.
Hidden 0.5, input 0.2, output zero.
Forgotten eval jitters 0.62 to 0.88.