At the heart of attention: each token forms a QUERY (what am I looking for?) and every token offers a KEY (what do I have?). Dot the query against each key and the biggest match is where the gaze goes — the query ‘attracts’ the most similar key. A scaling by 1/√d keeps the numbers from blowing up before the softmax. Slide the query and watch which token it locks onto.
Attention scores are a scaled dot product: score(q, k) = q·k / √d, computed between a query vector and each key. The dot product measures alignment — a query attracts the key pointing most in its direction — and the 1/√d scaling counteracts the growth of dot-product magnitude with dimension d, keeping the scores in a range where softmax has useful gradients (without it, large scores saturate softmax into a near-hard max too early). The scores then pass through softmax to become attention weights. A fail-loud self-check throws unless the query attends most to its most-similar key. ◆ real ML mechanism, node-verified.
A low-dimensional QK illustration (the exact scaled-dot-product scoring and its most-similar-key selection); real Q and K are learned linear projections of the residual stream — the alignment-scoring mechanism is exact.