A trick to run a big model faster: a small, cheap DRAFT model proposes several tokens at once, and the big TARGET model verifies them all in one pass — accepting the ones it agrees with and correcting at the first it doesn’t. When the draft agrees, you get many tokens for the price of one check; the output distribution is provably identical to the target’s. Slide the draft’s quality.
Speculative decoding: a draft model samples xᵢ ~ q, and the target (p) accepts each with probability min(1, p(xᵢ)/q(xᵢ)); on the first rejection it resamples from the normalised positive residual (p−q)⁺. This yields samples EXACTLY from the target distribution p (Leviathan et al. / Chen et al. 2023) while running the expensive target only once per block — so the speedup grows with how often the cheap draft agrees. A fail-loud self-check throws unless an identical draft is always accepted while a divergent draft is sometimes rejected. ◆ real decoding, node-verified.
The accept/reject is modelled on a fixed 4-token distribution; the min(1,p/q) rule and its exactness (samples ~ p) are the real, proven content. Actual speedup depends on draft quality and hardware — the on-screen figure is illustrative.