Each token asks a question and every other token answers with a score; the answers are turned into a set of weights that sum to one, and the token walks away carrying a weighted blend of what it heard. That is scaled dot-product attention — output = softmax(QKᵀ/√d)·V — the one operation the whole Transformer is built from, and it is runnable. Down the center, data flows: the matrices go in, the engine computes, the new representations come out. The blue team builds and defends it; the red team tries to break it.
source Vaswani et al., Attention Is All You Need (2017), §3.2.1 — arxiv.org/abs/1706.03762. Rendered, not quoted.
Nothing is memorized; the output falls out of four matrix steps:
1 score every query against every key: S = Q·Kᵀ. 2 scale by 1/√d so the dot products do not blow up as the dimension grows (large scores push softmax into vanishing gradients). 3 per row, softmax turns scores into weights that are non-negative and sum to 1. 4 the weights average the value rows: O = W·V.
Live scores S = Q·Kᵀ/√d for the current matrices (pre-softmax):
| q\k |
|---|
A neighbouring sphere, the-word-embedding, is the premise: it turns each token into a vector — a point in meaning-space. But an embedding sits still; "bank" cannot yet tell a river from a vault.
Attention is the next step: it lets those vectors look at each other. Q, K and V are all learned projections of the same embeddings, so a token can pull in context from its neighbours and become a contextual vector. Each sphere is the next one's input.
The blue team's live check: recompute the weights from the current matrices and confirm every row sums to 1 and every weight lies in [0,1], plus a fixed self-attend probe. If red tampers, this badge is where it shows.
Three tokens, each a row. From every token's embedding the layer projects three vectors of dimension d = 2: the query (what am I looking for?), the key (what do I offer?), and the value (what I actually pass on). Query·Key measures fit; the fit decides how much of each Value a token collects.
Edit any number below and the panel recomputes on the spot — nothing is looked up.
Each row of the heatmap is one query token's attention over the three keys; brighter = more weight; every row is normalized to sum to 1.
What the machine produces, proven: for each token a new vector O = W·V — a weighted blend of the value rows, the token's context-aware representation handed to the next layer.
| token | o₀ | o₁ |
|---|
The blue team's witness (left) confirms the weights are a real distribution; the red team (right) tries to make them lie.
It also has no built-in notion of order: permute the input rows and the output permutes with them — position must be bolted on separately (positional encodings). And stacked deep, attention can suffer rank/entropy collapse — heads drift toward uniform or identical maps and stop adding information. Attention is powerful; it is not free and it is not complete.
"The attention weights explain the model's decision." Cut. Weights are one factor, not a faithful attribution — Jain & Wallace (2019) showed different weightings can yield the same output. High weight ≠ cause.
"The Transformer invented attention." Cut. Bahdanau et al. (2014) introduced attention for translation; this paper's move was removing recurrence and keeping only attention.
"Softmax normalization is optional." Kept, corrected. Linear/kernel attention variants exist — but drop the row-sum here and the weights stop being a distribution and the output is wrong, which is exactly the tamper below.
The red team's move: skip the softmax normalization — take raw exp(scores) and never divide by the row sum. Weights no longer sum to 1; the average is corrupted and the output is wrong. The blue team's witness (window 7) is watching.
Remove the divide-by-sum and the rows stop being a probability distribution — the witness recomputes, finds row sums ≠ 1, and turns red. Nothing is faked; the attack is real and it is caught.