Linear Regression (OLS)

Linear regression is the foundation of predictive modeling. Given a set of points (like square footage and house price), we want to find the single straight line that best captures the trend.

But how do we define "best"? We define it by looking at the errors: the vertical distance from each data point to our line. If we square these errors (which heavily penalizes points that are far away from the line) and average them, we get the Mean Squared Error (MSE).

The goal of Linear Regression is to find the slope and intercept that make this MSE as small as possible. Because squaring the errors creates a smooth, bowl-shaped mathematical surface, we can actually use calculus to jump directly to the exact answer without guessing. This direct mathematical solution is called Ordinary Least Squares (OLS).

Try dragging the line toward just two nearby points until it passes through them exactly. The fit looks perfect for those two — and the total squared error across every other point gets much worse. That tension, between fitting a few points closely and fitting all of them reasonably, is exactly what squaring and averaging resolves in one step: OLS does not favor any single point, it finds the one line that minimizes the sum of every squared error at once.