Total Derivative

A partial derivative freezes everything but one input. The total derivative unfreezes them all and reports what actually happens when everything moves.

With t driving both x and y into their product, the x route adds 12 and the y route adds 4, so the total rate 16 dwarfs the frozen partial 3

Why Does This Exist?

Partials lie by omission, politely. The partial $\partial f / \partial x$ tells you what happens when $x$ moves and nothing else does. In any model worth training, nothing else stays still: $x$ and $y$ both come from shared upstream parameters, so nudging one dial turns several gears. Optimizers, sensitivity analyses, and hyperparameter studies need the honest number that counts every gear.

The total derivative is that number: each dependency is a route, each route contributes, and the total is the sum. Confusing it with a partial is one of the quietest bugs in ML mathematics, because the wrong answer looks like the right one with a smaller value.

Place it among its four neighbours before using it. Derivatives and partial derivatives teaches the frozen atoms, one slope with everything else held still. The single-variable chain rule multiplies slopes along one chain with no branches. The multivariable chain rule is the tree sum itself: one input through several routes, multiply along each and add. Multivariable calculus is the hub map of all four tools and never computes a route sum. This page is the accounting question: when a frozen partial answers and when only the all-routes total does.

The one idea you need first

A partial derivative $\partial f / \partial x$ holds all other direct inputs fixed. The total derivative $df / dt$ holds nothing fixed: it follows $t$ through every variable $t$ touches. Same function, different question. One asks about an isolated nudge, the other about a coordinated move, and only the second describes what happens during training.

Think of It Like This

Rent when both price and flat size change

Your rent is price per square metre times area. The landlord raises the unit price and you also move to a bigger flat in the same month. Asking how much of the bill change came from the price alone is the partial question. Asking what your new bill actually is needs both changes added.

The total derivative does that full accounting. It takes the price route on the old area plus the area route at the old price and adds them. Skip either route and your budget is wrong in exactly the direction you ignored.

The estimate stops working for large moves, where the interaction the linear sum drops starts to matter.

How It Actually Works

For $f(x, y)$ with $x(t)$ and $y(t)$ both functions of $t$ :

\frac{df}{dt} = \frac{\partial f}{\partial x}\frac{dx}{dt} + \frac{\partial f}{\partial y}\frac{dy}{dt}

In plain words: $t$ pushes $f$ through $x$ and through $y$ , and the total push is both pushes added. When only one variable depends on $t$ , the other term is zero and the total collapses to a single product. When neither does, both vanish, which is the formal way of saying a constant does not move.

The differential form says the same thing for small steps: $df = f_x\, dx + f_y\, dy$ . Each term pairs a frozen sensitivity with the actual movement of its variable. Gradient descent lives on this identity: the loss change over a step is approximately the gradient dotted with the step, which is the total derivative along the step direction. Read the identical sum as multiply-along-each-route-and-add and it is the multivariable chain rule; the arithmetic never changes, only the question. The matrix form of the same sum is the Jacobian and Hessian.

Worked example

Take $f(x, y) = xy$ with $x = t^2$ and $y = t + 1$ , at $t = 2$ .

Position first: $x = 4$ , $y = 3$ , $f = 12$ . The frozen partial $\partial f / \partial x = y = 3$ . Stopping there predicts that raising $t$ nudges $f$ gently.

The total says otherwise. Via $x$ : $y \cdot dx/dt = 3 \times 4 = 12$ . Via $y$ : $x \cdot dy/dt = 4 \times 1 = 4$ . Total: $12 + 4 = 16$ , more than five times the partial. Direct check: $f(t) = t^2(t+1) = t^3 + t^2$ , so $f'(t) = 3t^2 + 2t = 12 + 4 = 16$ . The partial missed the $y$ route entirely.

Code

Two routes to the same number: the route-by-route total and the substituted direct derivative.

1def total_rate(t: float) -> float:2    x = t**23    y = t + 14    return y * 2 * t + x * 1  # x route plus y route5
6
7def direct(t: float) -> float:8    return 3 * t**2 + 2 * t  # from f(t) = t^3 + t^29
10
11print(total_rate(2.0))  # -> 16.012print(direct(2.0))      # -> 16.0

Both print 16. The first line inside total_rate is the definition in code: each term is one dependency route, and deleting either term reproduces the classic bug.

Watch Out For

Reporting the partial as the answer in a coupled system

In a network every activation depends on shared weights, so almost no variable moves alone. Quoting $\partial f / \partial x$ where $df / dt$ belongs understates sensitivities, misranks feature importance, and mistunes step sizes. The symptom is a sensitivity analysis that disagrees with finite differences. The fix is mechanical: list every variable the input touches and sum the routes.

Detaching a variable and deleting its route in code

Calling .detach() or stop_gradient on a tensor severs a dependency route: the value flows forward but no gradient flows back through it. Used deliberately it implements fixed targets and alternating optimization. Used accidentally on a tensor reused downstream, it silently zeroes a term of the total derivative and the model trains on a censored gradient. If a loss stops responding to a parameter it used to track, check for a stray detach before doubting the architecture.

The Quick Version

The partial freezes every other input; the total follows the parameter through all of them.
Total equals the sum over dependency routes of frozen partial times variable rate.
For $xy$ with both factors driven by $t$ , the total 16 dwarfs the partial 3.
The chain rule is one chain, the multivariable chain rule is the tree sum, and this page is when to use that sum.
The differential $df = f_x dx + f_y dy$ is the same identity for small steps.
Detaching a tensor in code deletes a route from the total, silently and by design or by accident.