Core ML

Neural Networks (Multi-Layer Perceptron)

A stack of simple math units that learns to turn raw inputs into useful predictions by bending straight lines into curves.

Inputs on the left flow through a hidden layer of neurons to a prediction on the right

Why Does This Exist?

Plain linear models, the kind that fit a straight line through your data, are fast and easy to reason about. The catch: they can only ever draw straight boundaries. Ask one to separate photos of cats from dogs, or to model how house price bends with size and neighborhood, and it hits a wall. Real relationships curve, and a straight line can't follow them.

A multi-layer perceptron (MLP) fixes this by stacking many tiny decision units into layers and slipping a nonlinear step between them. That small addition lets the network bend and fold its decision boundary into almost any shape. It's the original "deep" neural network, and it still lives inside most modern models as a building block.

Think of It Like This

An assembly line of feature-builders

Picture an assembly line. The first station takes the raw parts (your input numbers) and combines them into simple sub-assemblies: "is there an edge here?", "is this value high?". The next station snaps those into bigger pieces, and the last one inspects the result and makes the call. Each station is dumb on its own, but stacked together they build up understanding one step at a time.

How It Actually Works

The atom of an MLP is a neuron. It does three small things: multiply each input by a weight, add everything up (plus a bias), then pass the result through an activation function.

Inside a single neuron

Step 1 of 3

Weighted sum

Multiply each input by its weight and add them together, plus a bias term that shifts the total.

Show all steps

Weighted sum: Multiply each input by its weight and add them together, plus a bias term that shifts the total.
Activation: Pass that sum through a nonlinear function like ReLU, which keeps positive values and zeroes out the rest.
Stack into layers: Dozens of neurons form a layer, and layers feed the next — data flowing straight through is the forward pass.

Weighted sum. Each input is scaled by a weight that says how much it matters, and the results are added together with a bias term that shifts the total up or down.
Activation. The sum is fed through a nonlinear function like ReLU, which simply keeps positive values and zeroes out the rest. This is the step that lets the network represent curves.
Stack into layers. Dozens of neurons form a layer; layers feed into the next. Data flowing straight through, input to output, is called the forward pass.

The nonlinearity is the secret sauce. Without it, ten stacked layers collapse into the math of a single straight-line model. With it, a network with enough neurons can approximate essentially any continuous function. That's the universal approximation idea, and it's why even a modest MLP is surprisingly capable.

Show Me the Code

1import numpy as np2
3
4def relu(x: np.ndarray) -> np.ndarray:5    return np.maximum(0, x)  # keep positives, zero out the rest6
7
8def mlp_forward(9    x: np.ndarray, w1: np.ndarray, b1: np.ndarray, w2: np.ndarray, b2: np.ndarray10) -> np.ndarray:11    h = relu(x @ w1 + b1)  # hidden layer: mix inputs, then bend the line12    return h @ w2 + b2  # output layer: a linear score13
14
15x = np.array([[0.5, -1.2, 3.0]])  # one sample, three features16w1, b1 = np.random.randn(3, 4), np.zeros(4)17w2, b2 = np.random.randn(4, 1), np.zeros(1)18print(mlp_forward(x, w1, b1, w2, b2))  # -> one prediction

That's the whole forward pass: two matrix multiplies with a nonlinear bend in the middle. Training is what sets good values for those weights.

Watch Out For

Stacking linear layers gains you nothing

Without a nonlinear activation between layers, ten layers collapse into the math of one. The activation is the thing that makes depth actually matter.

Bigger isn't automatically better

Piling on layers and neurons can overfit (memorize the training data) and makes gradients harder to push through during training. Grow the network to fit the problem, not your ambition.

The Quick Version

An MLP stacks layers of simple neurons with a nonlinear step between them.
Each neuron computes a weighted sum, adds a bias, then applies an activation like ReLU.
That nonlinearity is what lets the network model curves, not just straight lines.
Stacked layers build features from simple to complex.
It learns its weights through backpropagation.