Exponential and Logarithmic Functions

Exponential functions grow by multiplying, logarithmic functions grow by counting multiplications. One models spread and decay, the other reads it back on a human scale.

Two to the x climbs through (3, 8) doubling every step while log base 2 of x strolls through (8, 3), each function undoing the other

Why Does This Exist?

Two shapes run the quantities ML cares about. Populations, virality, compounding returns and exploding gradients grow exponentially: multiply per step, and the curve goes vertical. Measurement, bits, magnitudes and learning curves grow logarithmically: count the multiplications, and thousand-fold spans fit on one axis.

They are inverses, mirror images across the line $y = x$ . Exponentials answer "how much after $t$ steps"; logs answer "how many steps to reach $N$ ". The log rules themselves are in logarithms.

The one idea you need first. $2^3 = 8$ says three doublings reach eight. $\log_2(8) = 3$ counts them back. Growth multiplies; the log counts the multiplications. Hold both directions at once and the pair is simple.

Think of It Like This

A rumour and its contact tracer

A rumour spreads exponentially: each hearer tells two friends, so 1 becomes 2, 4, 8, doubling per round. The contact tracer works logarithmically: given 8 hearers, how many rounds ran? Three. The tracer counts doublings the way the rumour performs them.

Neither job replaces the other. The rumour predicts load; the tracer reconstructs history. Confuse them and you either expect linear spread from a compounding process or demand round-by-round logs from thin air.

Where it stops: rumours saturate when everyone has heard, while pure exponentials never do. Real growth is logistic; the exponential is its early phase, which is exactly the phase ML usually models.

How It Actually Works

Exponential form: $f(x) = b^x$ with base $b > 0$ , $b \neq 1$ . Growth for $b > 1$ , decay for $0 < b < 1$ . The special base $e \approx 2.718$ makes calculus clean, since $e^x$ is its own derivative. Doubling time $T$ satisfies $b^T = 2$ ; with 10 percent growth per period, $1.1^T = 2$ gives $T = \ln 2 / \ln 1.1 \approx 0.6931 / 0.0953 \approx 7.3$ periods. Half-life mirrors it: $(1/2)^n$ remains after $n$ half-lives.

Logarithmic form: $g(x) = \log_b(x)$ , defined for $x > 0$ , the exact inverse. It climbs steeply near zero then flattens: from 1 to 10 takes as much horizontal room as 10 to 100, each decade one unit on a base-10 log. Small inputs near zero plunge toward negative infinity, which is why log-scales never touch the axis.

Worked example

Culture: $500$ cells doubling every $2$ hours. After $6$ hours, three doublings: $500 \times 2^3 = 500 \times 8 = 4000$ cells.

Half-life: $100$ g with a $2$ -hour half-life. After $6$ hours: $100 \times (1/2)^3 = 100/8 = 12.5$ g.

Count: $\log_2(32) = 5$ , because $2^5 = 32$ . Five doublings from 1 reach 32, and the log counted them. The diagram above draws both directions: the climb through $(3, 8)$ and the stroll through $(8, 3)$ .

Code

1cells: int = 5002for _ in range(3):3    cells *= 24print(cells)  # -> 40005
6import math7print(math.log2(32))  # -> 5.08print(100 * (0.5) ** 3)  # -> 12.5

The loop performs the doublings one by one, the honest version of exponentiation. The log line counts them back. The last line is the half-life in one expression: three halvings of 100.

Watch Out For

Exponentials overflowing to infinity

$e^{1000}$ is about $10^{434}$ , far past float64's ceiling near $10^{308}$ , so softmax-style code exponentiates straight to inf and then divides inf/inf into NaN. The symptom is a loss that is fine for epochs, then NaN on one unlucky batch with large logits.

Subtract the max before exponentiating: exp(x - max(x)). Work in log-space wherever the math allows. Overflow is not a rare edge; it is the default fate of raw exponentials.

Logging zero in a measurement pipeline

$\log(0)$ is negative infinity, and any zero count, empty bin or exact-zero probability feeds it directly. Dashboards show -inf, downstream means become NaN, and the pipeline stays green because no exception fired.

Add epsilon before the log or filter zeros explicitly: math.log(x + 1e-12). Decide the floor on purpose instead of discovering negative infinity in a report.

The Quick Version

$b^x$ multiplies per step: growth above 1, decay below it, $e^x$ for calculus.
$\log_b(x)$ counts the multiplications; defined only for positive $x$ .
Doubling triples 500 to 4000 in three rounds; three half-lives cut 100g to 12.5g.
Raw exponentials overflow past $\sim 10^{308}$ : subtract the max first.
Never log raw zeros; set an epsilon floor deliberately.