A model that writes one word at a time must NOT be allowed to peek at the words that come after — that would be cheating, seeing the answer. The causal mask enforces it: before the softmax, every ‘look at the future’ score is set to minus-infinity, so its weight becomes exactly zero. Each token attends only to itself and what came before. Slide the position and watch the future go dark.
Autoregressive language models predict the next token from the previous ones, so during attention a token at position i must not attend to positions j > i. The CAUSAL (look-ahead) MASK adds −∞ to every future score before softmax, so e^(−∞) = 0 and those tokens receive exactly zero attention weight; the remaining weights over positions ≤ i still sum to 1. It is a lower-triangular mask, and it is what lets a transformer be trained in parallel over a whole sequence while still, at every position, only using the past — the same as generating left-to-right. A fail-loud self-check throws unless attention to future positions is zero and the visible weights sum to 1. ◆ real ML mechanism, node-verified.
The decoder causal mask (the exact zero-to-the-future, sum-to-one-over-the-past behaviour); encoders and bidirectional models omit it — the no-peeking-ahead constraint is exact for autoregressive generation.