THE PSEUDOINVERSE the best answer when there is no exact one — invert the un-invertible
Not every system has a solution. Too many equations and none fits exactly; too few and infinitely many do. The Moore–Penrose pseudoinverseA⁺ gives the best answer in both cases: for an overdetermined system it returns the least-squares fit (minimizing the residual); for an underdetermined one it returns the minimum-norm solution. It is the closest thing to A⁻¹ when A⁻¹ does not exist, defined for every matrix by four clean axioms. Reconstructing an estimate from imperfect or incomplete measurements — regression, control, tomography, machine learning — is, at bottom, applying a pseudoinverse.
THE TECHNIQUE A⁺: least-squares if overdetermined, minimum-norm if under
Three noisy readings of one quantity — an overdetermined system [1;1;1]x=[1;2;3]. The demo applies the pseudoinverse to reconstruct the best estimate: live demo
HISTORY & CREDIT Moore 1920 · Penrose 1955
“If a system has no solution, there is nothing to compute.” — the pseudoinverse computes the best one anyway: nearest fit when over-constrained, smallest solution when under. Invert what cannot be inverted. cited
1920 · E. H. Moore — the general reciprocal of a matrix. 1955 · Roger Penrose — the four defining conditions; Bjerhammar (1951) independently for least squares. now · the engine under regression, least-squares estimation, and the normal equations (dart 305).
Four axioms define a reciprocal for every matrix — least-squares one way, minimum-norm the other. The best reconstruction where an exact one is impossible. Moore-Penrose
RECOMMEND FOR I-13 least-squares estimate from noisy readings, computed
On the canonical compiler, three readings 1,2,3 of one quantity reconstruct to the least-squares estimate 2 — their mean, which the pseudoinverse of [1;1;1] delivers:
$ i13 run rec_pseudoinverse.i13 # A+ of [1;1;1] applied to [1;2;3]
xhat = 2 -- the least-squares (minimum-residual) reconstruction: the mean
Recommend: the pseudoinverse is reconstruction when exactness is impossible, and here i13 computes its simplest instance: the least-squares estimate of one quantity from three noisy readings is the mean, 2. The structure that makes this the right reconstruction is the least-squares geometry — the estimate is the orthogonal projection of the data onto the column space. It generalizes the batch's exact recoveries to the noisy, over-constrained world: where interpolation (dart 294) demanded the data lie exactly on a low-degree curve, the pseudoinverse finds the closest such curve when it does not. Recovery of a best estimate, caused by the projection structure.