Joint, Marginal and Conditional Distributions

The joint table holds everything. Add across a row to ignore a variable, or zoom into one row to learn from it. Those two moves run all of inference.

A two by two joint table with row sums 0.6 and 0.4, and the Y equals 1 column rescaled by 0.4 into the conditional split 0.25 and 0.75

Why Does This Exist?

Data arrives joint and questions arrive sliced. Sensors report feature and label together, but deployment asks for the label given the feature. Trials record treatment, recovery, and age together, but policy asks about recovery given treatment. The joint distribution holds every answer at once, and two moves extract the one needed.

Those moves divide the field. Summing a variable out ignores it, which removes nuisances and computes Bayes theorem denominators. Conditioning zooms into what was observed, which is what every classifier outputs: probability foundations defines a classifier as exactly such a conditional. Mixture models, EM, and Gibbs sampling are choreography of these two steps.

The one idea you need first

A joint chance $P(X = x, Y = y)$ is the fraction of outcomes where both hold. Fix $Y = y$ and re-normalize the survivors to total one for the conditional $P(X \mid Y = y)$ . Add the joint over all $y$ instead and $Y$ disappears for the marginal $P(X)$ . Slice versus sum, zoom versus ignore.

Think of It Like This

A school roster and two questions

The roster lists every student with grade and sport. Ask what fraction plays soccer and you scan the whole roster for the marginal. Ask what fraction of tenth-graders plays soccer and you first filter to tenth grade, then scan within for the conditional.

Filtering before counting is the entire difference. The roster never changes, but the question selects its audience. Counting filtered players against the whole school understates by the filter share, which is every conditioning bug in one line.

The analogy stops at observation. Rosters are complete and exact, while real joints are estimated from samples, so sums and slices inherit sampling noise a roster never has.

How It Actually Works

Three objects and two moves. The joint $P(X, Y)$ weighs every combination. The marginal removes a variable by the sum rule:

P(X = x) = \sum_y P(X = x, Y = y)

In plain words: add the row, because every value of $Y$ was counted. The conditional zooms by re-normalizing a slice:

P(X = x \mid Y = y) = \frac{P(X = x, Y = y)}{P(Y = y)}

In plain words: keep only the $y$ outcomes and stretch them to total one. Continuous variables replace sums with integrals, and the expectation versions average against these sliced weights.

Two facts organize practice. Marginals can match while joints differ wildly, so passing marginal checks never validates a joint model. The average of conditionals is the marginal, but nonlinear summaries need not agree: the mode of conditionals need not be the marginal mode.

Worked example

Take $P(0, 0) = 0.5$ , $P(0, 1) = 0.1$ , $P(1, 0) = 0.1$ , $P(1, 1) = 0.3$ .

Marginals by summing: $P(X = 1) = 0.1 + 0.3 = 0.4$ and $P(Y = 1) = 0.1 + 0.3 = 0.4$ . Row sums are 0.6 and 0.4, column sums are 0.6 and 0.4, a symmetric profile.

Conditionals by slicing: given $Y = 1$ , the column total is $0.4$ . Then $P(X = 1 \mid Y = 1) = 0.3 / 0.4 = 0.75$ and $P(X = 0 \mid Y = 1) = 0.1 / 0.4 = 0.25$ . Knowing $Y$ nearly doubled the belief from 0.4 to 0.75, which is dependence: the slice disagrees with the profile. Agreement at 0.4 would have meant independence.

Code

The same two moves in code, with the re-normalization beginners forget made explicit.

1joint = {(0, 0): 0.5, (0, 1): 0.1, (1, 0): 0.1, (1, 1): 0.3}2
3
4def marginal_y(y: int) -> float:5    return joint[(0, y)] + joint[(1, y)]  # sum the column away6
7
8def cond_x_given_y(x: int, y: int) -> float:9    return joint[(x, y)] / marginal_y(y)  # slice, then re-normalize10
11
12print(marginal_y(1))  # -> 0.413print(round(cond_x_given_y(1, 1), 2))  # -> 0.75

The prints give 0.4 and 0.75, matching the hand computation. The code never divides by anything but the slice total, which is the standing check: conditionals must sum to one.

Watch Out For

Conditioning without re-normalizing

Reading raw joint values after slicing understates every chance by the slice share. Here the column sums to 0.4, so raw readings run two and a half times too small.

Always divide by the slice total and assert the result sums to one as a standing check.

Marginalizing away the variable the question is about

Summing out the target feels productive because the table shrinks, but the answer lived in the removed dimension. Compute the marginal of what you observe and the conditional of what you want.

Reversing them answers how likely the evidence is rather than how likely the hypothesis is, which is the prosecutor fallacy in one line.

The Quick Version

The joint weighs every combination, and everything else derives from it.
Marginalization sums a variable away and ignores it, while conditioning slices to observations and learns from them.
Conditionals must re-normalize by the slice total, or every answer is short by the slice share.
In the worked table, Y equal to 1 lifts X to 0.75 against a marginal of 0.4, which is dependence.
Matching marginals never validate a joint, because different togetherness hides behind identical profiles.