Give it a cloud of points and it draws the one straight line that no other line can beat — the line whose squared misses sum to the smallest number possible. Drag the points, watch the line chase them, read the residuals it is trying to shrink. This is the closed form Gauss and Legendre wrote down: not a search, a solved equation.
source Gauss, Theoria motus corporum coelestium (1809) · Legendre, Nouvelles méthodes (1805) — archive.org/details/bub_gb_ORUOAAAAQAAJ
Rendered, not quoted.
Fit y = m·x + b to n points. Define the residual of a point as its vertical miss, e = y − (m x + b). Choose (m, b) to minimise SSE = Σ e².
Setting the two partial derivatives to zero gives the normal equations, solved in closed form:
m = (n·Sxy − Sx·Sy) / (n·Sxx − Sx·Sx)
b = (Sy − m·Sx) / n
where Sx=Σx, Sy=Σy, Sxx=Σx², Sxy=Σxy. No iteration, no guessing — one evaluation lands on the exact minimum.
This is the ancestor of every regression. The SSE surface here is a perfect paraboloid with one bottom; the closed form jumps straight to it.
When the model is not linear you cannot solve it in one line — so you roll downhill on the same SSE loss instead. That downhill roll is gradient descent, and the quantity it minimises is exactly this squared-error loss. Least squares is the seed; the whole training arc grows from it.
A live re-check. It refits the fixed dataset (1,1)(2,2)(3,3) from the engine right now and confirms it recovers m=1, b=0. If the engine is tampered, this recomputes a wrong slope and flips red.
A set of 2D points. Click empty canvas to add one; drag a point to move it; the presets load known clouds. Everything below is computed from these coordinates — nothing is baked.
The fitted line (cyan) with each residual drawn as a dropped segment from point to line. The reported SSE is the minimum reachable — no other (m, b) makes it smaller. That claim is enforced live by the self-check.
Least squares minimises vertical error and assumes x is exact. It is not symmetric: swap x and y and you get a different line. It also has no unique answer when all x are equal — the denominator n·Sxx − Sx·Sx goes to zero and the slope is undefined (a vertical cloud has no y=mx+b fit).
And it is famously not robust: because misses are squared, a single far-off outlier can drag the whole line toward it. The line is honest about the loss it minimises, not about which loss you should have wanted.
“Least squares gives the true line the data came from.”
Correction: it gives the line that minimises squared vertical residuals for this sample. That estimate is unbiased under specific noise assumptions (Gauss–Markov), but it is not the “true” line — different samples give different fits.
“A lower SSE always means a better model.”
Correction: SSE always drops as you add parameters; driven to zero it just memorises. Fit quality is a separate question from the minimisation this engine performs.
The disclosed planted void. This drops the − Sx·Sx term from the denominator, so the slope formula becomes (n Sxy − Sx Sy)/(n Sxx) — wrong, and no longer the SSE minimum. On (1,1)(2,2)(3,3) it returns m≈0.143 instead of 1.
Watch THE WITNESS (window 7) catch it and go red the instant you press it.
THE LEAST SQUARES · WORLD II · THE FOLD · refined OCHO. The engine computes the closed-form fit live from the points on the canvas; the self-check throws if the minimum property, the collinear recovery, or the known dataset ever fail, and the planted void is proven to make noise before it is restored. Built by David Lee Wise (ROOT0) with AVAN.