Give a cloud of points its own axes. The direction of greatest spread is the first principal component; the next, at right angles, carries what is left. Pearson found them in 1901 as the lines of closest fit — and they are runnable: form the covariance matrix, solve a 2×2 eigenproblem in closed form, and the axes fall out. Down the center the data flows: points go in, the engine decomposes, the axes and the variance captured come out. The blue team builds and defends it; the red team tries to break it.
source Pearson, K. (1901) “On Lines and Planes of Closest Fit to Systems of Points in Space,” Philosophical Magazine 2(11):559–572 — scholar.archive.org · pqdixezszjgqdjtdrt7eaxyre4. AMBER no dedicated archive.org/details item was located in search; scholar.archive.org resolves the scanned original. Rendered, not quoted.
Center the cloud on its mean. The spread is one symmetric matrix:
C = 1/N · Σ [ (x−x̄)² , (x−x̄)(y−ȳ) ; · , (y−ȳ)² ]
The principal axes are the eigenvectors of C; each eigenvalue λ is the variance along its axis. For 2×2 there is a closed form — no iteration:
λ = (T ± √((a−d)²+4b²)) / 2 · v = ( b , λ−a )
Live covariance of the current cloud:
Pearson (1901) asked for the line that minimises squared perpendicular distance to a cloud — and proved it is the axis of maximum variance. Same object, two doors.
Hotelling (1933) named it the principal component; the singular value decomposition made it the engine of every embedding and dimensionality reduction that followed — word vectors, PCA whitening, the low-rank core of a weight matrix. Each sphere is the next one’s premise: closest fit → eigenaxes → latent space.
The blue team’s live check: recompute the decomposition of the current cloud and confirm C·v₁ = λ₁v₁, that PC1 carries at least as much variance as PC2, and that the axes are orthonormal. If red tampers, this badge is where it shows.
The input is N points in the plane, drawn along a hidden major axis with adjustable tilt and elongation (a fixed-seed generator, so every run is reproducible). PCA never sees the hidden axis — it is recovered from the data alone.
Only two things are fed to the engine: the coordinates and nothing else — no labels, no target. That is why PCA is unsupervised: structure out of geometry.
Move any control — the covariance is reformed and the eigenproblem re-solved on the spot, never looked up. — PC1 — PC2 feet = projection onto PC1.
What the machine produces, proven: the two orthonormal principal axes, the variance λ along each, and the fraction of total variance PC1 captures — the honest measure of how much a 2D cloud collapses to 1D.
The blue team’s witness (left) re-derives these live; the red team (right) tries to make PC1 point the wrong way.
It is also scale-dependent: change the units of one axis and the “principal” direction moves — which is why real pipelines standardise first, an assumption, not a fact. And maximum variance can be pure noise: the loudest axis is not always the informative one.
“PC1 is the most important feature.” Cut. PC1 is the highest-variance direction. Importance depends on the task; a low-variance axis can be the one that separates the classes.
“Pearson invented PCA.” Cut, credited. Pearson (1901) gave the closest-fit line; Hotelling (1933) named and formalised the principal component. Two authors, one lineage.
“Eigenvectors are unique.” Kept, qualified. Their sign is free (v and −v are both valid), and when λ₁=λ₂ the axes are undetermined — a real degeneracy the engine flags.
The red team’s move: swap PC1 and PC2, so the label “greatest variance” is pinned to the smallest-variance axis. The blue team’s witness (window 7) is watching.
Swap the axes and the green line jumps to the short direction, the captured fraction falls below 50%, and the witness — which tests that PC1 truly holds the most variance — recomputes, disagrees, and turns red. Nothing is faked; the void makes noise.