POIĒTIKĒ · the grammar beneath the story · kept by THE POET

THE POINTWISE MUTUAL INFORMATION ◧ 2D · ◍ 3D · ◆ 4D · ◐ shadow · 👶 TAP

Which words BELONG together? ‘New York’, ‘strong tea’ — some pairs co-occur far more than chance would predict. Pointwise mutual information measures exactly that: the log-ratio of how often two words appear together versus how often they would if independent. High PMI marks a collocation; near-zero means they just happen to share a page. Slide the association.

◆ LIT▲ AMBER
◧ THE MEASURE · 2D
◍ WHICH WORDS BIND · 3D · together more than chance
◆ THE FOURTH · 4D · a tesseract turns
◐ THE SHADOW · one dimension down
👶 THE TODDLER CORNER — one fat tap
p(x,y)
p(x)p(y)
PMI (bits)
collocation?

◆ LIT — exact / checkable

Pointwise mutual information PMI(x,y) = log₂( p(x,y) / (p(x)·p(y)) ): the log-ratio of the joint probability to what independence predicts. PMI > 0 means x and y co-occur MORE than chance (a collocation / bound pair); PMI = 0 means independent; PMI < 0 means they avoid each other. It is the core association measure in lexicography and word-embedding theory (word2vec implicitly factorizes a shifted PMI matrix). A fail-loud self-check throws unless a bound pair gives PMI>0 and an independent pair gives PMI=0. ◆ real information theory, node-verified.

▲ AMBER — the figure

p(x),p(y),p(x,y) are set for the demo; the log-ratio is exact. Raw PMI over-rewards RARE pairs (small denominators), which is why practice uses PPMI or PMI² / add-k smoothing — a real caveat, noted not hidden.

POIĒTIKĒ: every story is an old shape wearing new words.  — THE POET
David Lee Wise / ROOT0 / TriPod LLC  ·  the poietike workshop, with AVAN