Core ML

Optimisation for Decisions

Machine Learning predicts the future. Optimization tells you exactly what to do about it. When you combine them, you transition from Descriptive Analytics to Prescriptive Analytics.

Machine Learning generates predictions (e.g., demand for products). An Optimization Solver takes those predictions, applies business constraints (e.g., budget, warehouse space), and outputs the mathematically optimal decision.

Why Does This Exist?

Machine Learning is phenomenal at answering questions like:

What will the demand for apples be tomorrow? (Regression)
Will this user click this ad? (Classification)
If I lower the price by $5, how much will sales increase? (Causal Inference)

But business leaders don't actually care about predictions. They care about decisions. If your ML model predicts that demand for apples will be high, but you only have $1,000 to spend, your warehouse is half full, and shipping costs are skyrocketing... how many apples should you buy right now to maximize profit?

ML cannot answer that question. Mathematical Optimization (a branch of Operations Research) exists to solve the decision problem. By combining ML predictions with hard business constraints, optimization solvers calculate the absolute most profitable action you can take.

Think of It Like This

Imagine you are packing a backpack for a hiking trip. Machine Learning is the weather forecast predicting a 90% chance of rain, and predicting that a tent will give you 50 "units" of happiness while a sleeping bag gives you 40 units. Optimization is the math that figures out exactly which items to pack to maximize your total happiness, strictly constrained by the fact that your backpack can only hold 20 pounds.

How It Actually Works

The combination of ML and Optimization is called Prescriptive Analytics. It usually involves a two-stage pipeline: Predict, then Optimize.

1. The ML Stage (Predict)

You train standard ML models (XGBoost, LSTMs, etc.) on historical data.

Example: An airline trains an ML model to predict exactly how many people will want to buy a ticket for a flight from NY to London on Tuesday.

2. The Optimization Stage (Prescribe)

You formulate a mathematical model with three components:

The Objective Function: What are we trying to maximize or minimize? (e.g., Maximize Total Ticket Revenue).
The Decision Variables: What are the actual levers we can pull? (e.g., Price of a coach ticket, Price of a first-class ticket).
The Constraints: What are the physical or business limits? (e.g., Total passengers $\le$ 200; First-class tickets $\ge$ 10; Price cannot change more than once per day).

The Solvers

Once you write down these equations, you feed them into an Optimization Solver.

If the equations are entirely linear, you use Linear Programming (LP) (e.g., the Simplex algorithm).
If your decisions must be whole numbers (e.g., you can't sell 0.5 of a seat), you use Mixed-Integer Programming (MIP).
Heavyweight commercial solvers like Gurobi or CPLEX, or open-source solvers like GLPK, use advanced heuristics to search through trillions of possible combinations to find the provably optimal decision in milliseconds.

Show Me the Code

Here is a simplified example of using an open-source solver (PuLP) to optimize a marketing budget based on ML propensity scores.

1import pulp2import pandas as pd3
4# Assume an ML model already predicted the LTV and Conversion Probability for 3 users5df = pd.DataFrame({6    'user': ['Alice', 'Bob', 'Charlie'],7    'expected_ltv': [1000, 500, 200],  # From ML Regression8    'prob_convert': [0.10, 0.40, 0.90], # From ML Classification9    'cost_to_target': [50, 20, 5]       # Marketing cost10})11
12# Our Objective: Maximize expected revenue.13# Our Constraint: We only have a $60 marketing budget.14
15# 1. Initialize the Optimization Problem16prob = pulp.LpProblem("Maximize_Marketing_ROI", pulp.LpMaximize)17
18# 2. Define Decision Variables (Binary: 1 if we target them, 0 if we don't)19target_vars = pulp.LpVariable.dicts("Target", df['user'], cat='Binary')20
21# 3. Define the Objective Function (Maximize: LTV * Prob * Target)22expected_revenue = [23    df.loc[i, 'expected_ltv'] * df.loc[i, 'prob_convert'] * target_vars[df.loc[i, 'user']]24    for i in df.index25]26prob += pulp.lpSum(expected_revenue)27
28# 4. Define the Constraint (Total cost <= $60)29costs = [df.loc[i, 'cost_to_target'] * target_vars[df.loc[i, 'user']] for i in df.index]30prob += pulp.lpSum(costs) <= 6031
32# 5. Solve the problem33prob.solve()34
35# Output the optimal decision36for user in df['user']:37    if target_vars[user].varValue == 1:38        print(f"Decision: Target {user}")39
40# Output will likely say: Target Bob and Charlie (Total cost: $25, Expected Rev: $380)41# It skips Alice because despite her high LTV, her cost is too high for the low probability.

Watch Out For

Garbage In, Garbage Out (GIGO). Optimization solvers assume the numbers you feed them are absolute facts. If your ML model's prediction is wildly inaccurate, the solver will perfectly optimize a decision for a reality that doesn't exist. Often, it's better to use Stochastic Optimization, which incorporates the uncertainty (confidence intervals) of the ML prediction directly into the constraints.

The Quick Version

Machine Learning is descriptive and predictive; it tells you what will happen.
Mathematical Optimization (Operations Research) is prescriptive; it tells you what to do about it.
You build an optimization model by defining an Objective Function, Decision Variables, and strict Constraints.
Solvers (using Linear Programming or MIP) calculate the exact configuration of decisions that maximizes the objective without violating the constraints.

Related concepts

Contextual Bandits