Assume the features are independent given the class. The assumption is false — words in a document are not independent — and the classifier works anyway. Down the center a document flows: word-presence goes in, the posterior is computed by P(c) × ∏i P(xi|c), the predicted class comes out. The blue team proves it equals the true Bayes posterior; the red team drops the prior and watches it break.
source M. E. Maron, Automatic Indexing: An Experimental Inquiry, Journal of the ACM 8(3), 404–417, 1961 (the "idiot Bayes" / probabilistic indexing origin) — doi:10.1145/321075.321084. No stable free full text; cited by journal+year AMBER. Rendered, not quoted.
Take logs and the product becomes a sum. The log-odds between any two classes is an affine function of the features:
log P(a|x)/P(b|x) = w₀ + ∑i wi·xi
Live weights, sports vs tech (computed from the model, never looked up):
| term | weight |
|---|
Naive Bayes is a linear classifier — a straight decision boundary in feature space, which is why it is fast, tiny, and hard to overfit.
The exact posterior is Bayes' rule: P(c|x) ∝ P(c)·P(x|c). The bottleneck is P(x|c) — the joint over all feature combinations, which explodes.
Naive Bayes adds one bold assumption: P(x|c) = ∏i P(xi|c). That single factorization turns an exponential table into a handful of numbers. It is the-bayes with its premise deliberately broken — and it is the baseline every text and spam model is measured against.
Re-runs the engine over all 24=16 documents and checks two invariants live: posteriors sum to 1, and the factored NB posterior equals the full-joint Bayes posterior exactly. If red drops the prior, this badge catches it.
A document is reduced to which of four words appear (Bernoulli bag-of-words): xi = 1 if word i is present, else 0. Three candidate classes, each with a prior and per-word likelihoods:
| class | P(c) | P(ball) | P(chip) | P(vote) | P(tax) |
|---|
Each P(word|c) is the chance that class emits that word. These are the model's parameters — the whole classifier is four rows of numbers, fed into the panel below.
Toggle any word — the log-posterior is recomputed from P(c)·∏ P(xi|c) on the spot, never looked up.
| class | log P(c) | log P(x|c) | log-post | P(c|x) |
|---|
What the machine produces, proven: the predicted class is the argmax posterior, and that posterior is computed two independent ways — factored (NB, log-space) and by normalizing the full 3×16 joint table — agreeing to 1e-9 on every one of the 16 documents.
The blue witness (left) confirms the agreement live; the red team (right) drops the prior to break it.
It survives because classification only needs the argmax, not the number. The ranking of classes is often right even when the probabilities are nonsense. Trust the label; never trust the confidence.
"The 0.98 posterior means 98% sure." Cut. Correlated features make the probabilities meaningless; only the argmax is defensible. Calibrate before you believe a number.
"Independence must roughly hold or it fails." Cut. It classifies well even under heavy dependence — the decision boundary can be right while the model of P(x|c) is wrong.
"A word never seen for a class kills it." Kept, corrected. A zero likelihood zeroes the whole product — real; fixed by Laplace smoothing (add-one), not by pretending it cannot happen.
The red team's move: drop the class prior P(c) — treat sports/tech/politics as equally likely when they are not (0.5 / 0.3 / 0.2). The blue witness (window 7) is watching.
Drop the prior and the rare class (politics) is over-predicted; the factored posterior no longer equals the true Bayes posterior. The witness recomputes, the exact match breaks, and the badge turns red. Nothing is faked; the attack is real and it is caught.