EfficientNet Compound Scaling

EfficientNet scales depth, width and resolution together with one knob, so each bigger model spends its extra budget where it helps instead of on depth alone.

Scaling depth, width and resolution together with 1.2, 1.1 and 1.15 doubles FLOPs per step and lifts B0 to 77 percent top 1 with 5.3M weights

Why Does This Exist?

Scaling one dimension stalls. Deeper-only networks gain little past 200 layers, wider-only networks memorize, and higher resolution alone feeds detail the network has no capacity to use. Teams scaled whatever was easiest and paid full price for shrinking gains.

EfficientNet from Tan and Le in 2019 scales all three together with one compound coefficient $\phi$ (phi): depth by $\alpha^{\phi}$ (alpha), width by $\beta^{\phi}$ (beta), resolution by $\gamma^{\phi}$ (gamma), under the constraint $\alpha \cdot \beta^2 \cdot \gamma^2 \approx 2$ so each step costs about twice the FLOPs. The B0 baseline uses mobile inverted bottleneck blocks with squeeze-excitation, and B1 through B7 walk $\phi$ upward. This page covers the scaling rule. Phone-grade blocks live in MobileNet and the residual backbone in residual networks.

The one idea you need first

Model size has three axes. Depth is layer count, width is channels per layer, resolution is input pixels. FLOPs count the arithmetic per forward pass and set the training bill. Doubling FLOPs doubles cost, so a scaling rule should buy the most accuracy per doubling, which is what the constraint below enforces.

Think of It Like This

Enlarging a photo lab, not one machine

A photo lab doubling output cannot just hire developers. It needs proportionally more enlargers for depth of process, wider paper stock for throughput, and finer-grained film for input quality. One without the others jams the line: developers idle while prints queue, or fine film wasted on coarse paper.

Compound scaling is the proportional order form. Depth-only scaling is hiring developers while the enlargers queue up untouched. It stops holding where one input is fixed: a lab fed only thumbnail negatives gains nothing from finer film, and likewise resolution scaling wastes FLOPs when the objects are tiny.

How It Actually Works

Grid search on the B0 baseline found $\alpha = 1.2$ , $\beta = 1.1$ , $\gamma = 1.15$ . The constraint checks out:

1.2 \times 1.1^2 \times 1.15^2 = 1.2 \times 1.21 \times 1.3225 \approx 1.92

Near 2, so $\phi = 1$ roughly doubles FLOPs: depth grows 20 percent, width 10 percent, resolution 15 percent, and the squared terms reflect that width and resolution each enter compute quadratically. B0 at 224 pixels grows to B4 at 380 pixels with $\phi = 4$ , about $2^4 = 16$ times the FLOPs for a large accuracy step instead of the 16-times-depth alternative that stalls.

Worked example

B0 is concrete: about 5.3 million weights and 390 million FLOPs at roughly 77.1 percent top-1 on ImageNet, against ResNet-50 at 25.6 million weights and 4.1 billion FLOPs near 76 percent. Same accuracy band at one fifth the weights ( $5.3 / 25.6 \approx 0.21$ ) and one tenth the compute ( $0.39 / 4.1 \approx 0.10$ ). Scaling to B4 multiplies the 390 million FLOPs by $2^4 = 16$ to about 6.24 billion, which is the number the snippet prints. Version 2 later replaced early MBConv blocks with fused convolutions because depthwise steps on large early maps pay high memory-access cost for little arithmetic.

Code

The snippet verifies the doubling constraint and the B4 arithmetic from the worked example.

1alpha, beta, gamma = 1.2, 1.1, 1.152print(round(alpha * beta**2 * gamma**2, 2))3# -> 1.924
5b0_flops = 390_000_0006print(b0_flops * (2 ** 4) // 1_000_000)7# -> 6240

The 1.92 sits just under 2, which is the doubling the rule promises per $\phi$ step. The 6240 is millions of FLOPs, about 6.24 billion for B4, sixteen times the B0 budget.

Watch Out For

Scaling resolution past what the objects support

Raising input to 600 pixels on data where objects span 30 pixels adds FLOPs with no new signal. The symptom is slower training and flat accuracy while the resolution multiplier climbs. Hold resolution fixed when objects are small and spend the budget on width instead.

Copying B7 training settings onto B0 fine-tuning

Large EfficientNets need RMSProp with strong AutoAugment and stochastic depth; small ones fine-tune best with lighter augmentation and shorter schedules. The symptom is a B0 that underfits under B7 regularization while a straight small-model schedule converges. Match the schedule to the model size being trained, not the family name.

The Quick Version

EfficientNet scales depth, width and resolution together so each doubling of FLOPs buys real accuracy.
Coefficients 1.2, 1.1 and 1.15 satisfy the doubling constraint at about 1.92 per step.
B0 matches ResNet-50 accuracy near 77 percent with 5.3 million weights against 25.6 million.
MBConv blocks with squeeze-excitation carry the efficiency; version 2 fuses early blocks for memory access.
Scale resolution only when objects are large enough to reward it.