Model Pruning
Removing weights or whole structures from a trained network, usually the smallest-magnitude ones first, to shrink it with minimal loss in accuracy.
Weights below a magnitude threshold get zeroed out after training, on the reasoning that they contributed the least to the network's output. Unstructured pruning removes individual weights for the best accuracy-per-sparsity ratio, while structured pruning removes whole channels or neurons so ordinary hardware can realize the memory and speed savings without special sparse-matrix support.
Fine-tuning the surviving weights after pruning is standard practice, since the remaining weights were trained assuming their now-removed neighbors were still contributing.