Given two labelled clouds of points, which single direction pulls them furthest apart? Not the line joining the means — Fisher's answer is w = Sw-1(μ1−μ2): rotate the mean-difference by the inverse within-class scatter, and the ratio of between-class to within-class variance is maximised. Data goes in as two classes, the engine solves for the axis, and a separated 1-D projection comes out. Blue builds it; red drops the whitening to break it.
source R. A. Fisher, The Use of Multiple Measurements in Taxonomic Problems, Annals of Eugenics 7 (1936) 179–188 — no clean archive.org scan of the paper was located (search: archive.org/search · multiple measurements taxonomic problems), so this citation is AMBER. Rendered, not quoted.
Score a direction d by the Fisher ratio J(d) = (d·δ)² / (dᵀSwd), where δ=μ1−μ2 and Sw is the pooled within-class scatter. Numerator = squared gap between projected means; denominator = spread inside the classes.
Maximising this Rayleigh quotient is a solved problem: w = Sw-1δ, with maximum J(w) = δᵀSw-1δ. The Sw-1 is a whitening: it un-stretches the clouds first, so the axis dodges the direction they naturally smear.
Live for the current data:
Fisher, 1936. Where an unsupervised projection (PCA) hunts the axis of greatest total variance — blind to the labels — the discriminant uses them: it maximises between over within. Same act, a projection to one dimension; opposite objective.
Under equal-covariance Gaussians this very w is the Bayes-optimal linear classifier — the discriminant is the decision rule. The label is the premise this sphere adds; the next sphere inherits it. Each sphere is the next one's premise.
The blue team's live check: re-solve on the current data and confirm no swept or random direction beats J(w), and the class means stay distinct. If red drops the whitening, w stops being optimal and this badge goes red.
Two 2-D clouds, four points each, generated deterministically: class ① around μ1=(+sep,0), class ② around μ2=(−sep,0), sharing one tilted covariance. The tilt is what makes the naive "aim at the other mean" answer wrong — the clouds smear along a diagonal, so the honest axis must turn away from it.
You feed the engine the raw points and their labels; nothing about the answer is supplied. Live points:
| class | mean | points (x,y) |
|---|
Gold dashed = the solved axis w. Big rings = class means, green dot = threshold, faint squares = each point's foot on w. Move a slider — w is re-solved from Sw-1δ on the spot, never looked up.
What the machine produces, proven: the direction w = Sw-1δ whose Fisher ratio J(w)=δᵀSw-1δ is the maximum over every direction — checked against 180 swept plus 240 random axes — the two means projecting to distinct points, and on the well-separated default data a threshold that classifies all eight points with zero error.
The blue team's witness (left) re-confirms optimality live; the red team (right) drops the whitening to make it lie.
With more features than samples Sw is singular and w is undefined without regularisation (shrinkage / pseudo-inverse). And it maximises mean separation, not a calibrated posterior probability — a great axis is not the same as an honest confidence.
"w points straight from one class mean to the other." Cut. That is the naive direction δ. The true w = Sw-1δ rotates it by the inverse within-class scatter — equal only when Sw∝I. It is exactly the tamper below.
"LDA is just PCA with labels." Cut. PCA maximises total variance ignoring labels; LDA maximises between/within using them — different objectives, generally different axes.
"LDA always separates the classes perfectly." Cut. It returns the optimal linear projection; if the classes are not linearly separable, zero error is impossible. Optimal ≠ separable.
The red team's move: aim w straight down δ and skip the Sw-1 whitening. On the tilted clouds a plain swept direction then beats it — the witness (window 7) is watching.
Drop the whitening and w becomes the naive mean-difference; its Fisher ratio falls, a sampled axis out-scores it, and the witness recomputes and turns red. Nothing is faked; the attack changes a real number and it is caught.