Skip to content
AI360Xpert
Glossary
Definition

Universal Approximation

The theorem that a sufficiently wide single-hidden-layer network can approximate any continuous function on a bounded domain to within arbitrary accuracy.

Proved for sigmoid units by Cybenko in 1989 and generalized since, the theorem says such a network exists for the target accuracy you name — on a compact domain, with a non-polynomial activation. It says nothing about how wide "wide enough" turns out to be, and nothing about whether gradient descent can actually find those weights.

That gap between existence and learnability is the whole reason depth still matters in practice: a target that a wide shallow network could represent in principle may need an unaffordable number of units, while a deeper network represents the same function with dramatically fewer parameters.