One attention pattern can only capture one kind of relationship at a time. So a transformer runs SEVERAL attention heads in parallel, each in its own slice of the vector space — one head might track the subject, another the previous word, another matching brackets. Their outputs are stitched back together. Many narrow specialists beat one generalist gaze. Slide the head count and split the dimensions.
Multi-head attention runs h attention operations in PARALLEL, each on a d/h-dimensional projection of the input, then concatenates their outputs back to d dimensions (Vaswani et al., 2017). Splitting the model dimension across heads costs nothing in total compute but lets each head specialise on a different relationship — empirically, interpretability finds heads for previous-token, induction, subject-verb agreement, syntax, and more (the very [[induction-head]] types this domain catalogues). The heads see the same tokens but attend by different learned Q/K/V, so the model looks many ways at once. A fail-loud self-check throws unless h heads of d/h dims recombine to exactly d. ◆ real ML mechanism, node-verified.
The split-project-concat structure (the exact dimension bookkeeping, d = h × d_head); which relationships each head learns is emergent and not guaranteed — the parallel-specialised-heads mechanism is exact.