Not every token is worth the whole model. The confident ones can cleave early — read the answer off a shallow layer and stop. Drag the confidence gate and watch the compute you save trade against the answers you break.
LIT the exit rule & every number below, computed live FIG a toy 12-layer confidence field WALL a real multi-exit model
The stack — one token cleaving out
Each layer refines the guess; the bar is the model's confidence in its current top token. The token cleaves out at the first layer whose confidence clears τ — everything above it is never computed for this token.
Live readout
Where the batch exits
The whole tradeoff — sweep τ from loose to strict
■ compute saved ■ agreement with the full-depth answer · the dashed line is your current τ. Loose gate → cheap but wrong more often; strict gate → faithful but you pay for depth you didn't need. The knee is the whole game.