Which words tell you what a document is ABOUT? Not ‘the’ and ‘of’ — they are everywhere. A word matters when it is frequent in ONE document but rare across all of them. TF–IDF scores exactly that, and it is how every search engine decides what your page is really about. Slide through the words and watch the meaningful ones light up.
TF–IDF = term frequency × inverse document frequency = (count of the word in a document) × log(total docs / docs containing the word). A word common in a document scores high on TF, but if it appears in EVERY document its IDF is log(1)=0, zeroing it out — so stopwords like ‘the’ get weight zero while distinctive terms rise. A fail-loud self-check throws unless a distinctive term (‘cat’, only in doc 1) outscores a universal stopword (‘the’, in all docs) which must land at exactly 0.
Three toy documents and raw counts here; real systems add sublinear TF scaling, smoothing, and length normalisation (and neural embeddings now compete). The tf × log(N/df) arithmetic and the stopword-goes-to-zero fact are exact (Spärck Jones 1972).