How do you reconstruct the best line through scattered points? Least squares: minimize the sum of squared residuals. The minimum has a clean geometric characterization — the residual vector is orthogonal to the space of fits — which gives the normal equations AᵀA x = Aᵀb. Solve them and you have the coefficients of the best fit. This is the workhorse of regression, curve fitting, and calibration: reconstructing the underlying trend from noisy data. The name is literal — “normal” means perpendicular — and the structure is that the best reconstruction is the projection of the data onto what the model can represent.
Three points, fit a line through the origin y=ax. The demo solves the normal equations for the best slope: live demo
“The best fit is a matter of taste.” — least squares makes it a projection: the residual perpendicular to the model, one unique answer from the normal equations. Geometry, not taste. cited
Minimize the squared residual and it comes out perpendicular to the fit — the normal equations. The best reconstruction is the projection; the geometry names it. Gauss-Legendre
On the canonical compiler, points (1,2),(2,4),(3,5) fit y=ax with slope a = Σxy / Σx² = 25/14 ≈ 1.786: