the transformer, the hype, and a mirage that won a best-paper award
The transformer (2017, "Attention Is All You Need") is just a very good function approximator — attention layers, matrix multiplies, gradient descent. Nobody seriously calls that emergent. The interesting claim is about what happens when you scale it up: that large language models suddenly can do things smaller ones can't, abilities nobody trained in on purpose. That's the thing worth poking. So: are those abilities emergent? And the honest answer starts with it depends what you measured.
A much-cited 2022 paper (Wei et al.) defined an emergent ability as one absent in small models and present in large ones, with two thrilling properties: sharpness (it flips from can't to can almost instantly with scale) and unpredictability (you can't see it coming). Plot task accuracy against model size and you get a flat line that suddenly rockets upward — multi-step arithmetic, word unscrambling, and dozens more.
It was catnip. It hinted at magic, at AGI around the corner, and at a real safety worry: if dangerous skills can appear without warning, scaling is a blindfolded walk.
In 2023, Schaeffer, Miranda & Koyejo offered a deflating alternative: the jump is often in the ruler, not the model. Press the toggle — it's the same model both times. Only the metric changes.
A harsh, all-or-nothing metric like exact-match is roughly per-token accuracy raised to the power of the answer length. So even while the model improves smoothly token by token, the score stays pinned near zero — until it crosses a threshold and shoots up. Swap in a continuous metric (partial credit, per-token probability) and the same curve is a gentle, predictable ramp. The authors showed over 92% of "emergent" tasks on a big benchmark used such metrics — and even conjured fake emergence in vision models by choosing cruel rulers on purpose.
The mirage paper is precise: it says the sharpness and surprise are often artifacts — not that scaling does nothing. Two things stay genuinely strange:
Scaling laws. Loss falls as a clean power law with size and data (Kaplan 2020; Chinchilla 2022). Predictable, not magic — but real.
In-context learning & grokking. Few-shot skill from nowhere; networks that suddenly generalize long after overfitting. Still not fully explained.
So "it's all an illusion" is as wrong as "it's magic." There are real qualitative shifts we can't yet predict from first principles. We just shouldn't mistake a steep S-curve through a thresholding metric for a law of nature.
"Emergence" is borrowed from complexity science, where it has two flavors. Strong emergence: genuinely new, irreducible powers the parts could never explain. Weak emergence: surprising, but in principle derivable from the rules. Large models are, at most, weak — everything they do follows, in principle, from matrix multiplies and the training data. No new physics. The word just smuggles in mystery and inevitability it hasn't earned.
"Emergent" too often means: "we didn't predict it, so let's call it profound."
Yes, with an asterisk the size of a data center. The capabilities are real. The suddenness is mostly in the metric and the eye. The unpredictability is largely our own ignorance, not the universe being coy. Call it weak emergence wearing a profound-sounding coat — pointing at behavior we genuinely can't fully forecast yet, dressed up as something more cosmic than it is.
Mostly a measurement mirage. Partly a real open puzzle. Entirely overclaimed. That's the honest lol.
Fair to both sides: the 2022 measurements were real, and "emergence" as a loose label for new-behavior-at-scale is still useful to many researchers. The 2023 rebuttal didn't show capabilities are fake — only that their famous sharpness and unpredictability are frequently metric-driven, plus too-small test sets. Don't read it as "emergence debunked"; read it as "emergence, deflated and clarified." The debate is genuinely live (2023–2025).
Other caveats: scaling laws predict loss, not which specific skills appear when; grokking and in-context learning remain real open questions; and "weak vs strong emergence" is my framing of a philosophy debate that has its own disputes. The chart above is an honest toy — a sigmoid skill curve, with exact-match modeled as that curve raised to a power — to show the mechanism, not real benchmark data.
And the meta-joke, paid off: a large language model wrote this pamphlet about whether large language models are emergent, using the Japanese word for emergence — 創発 — to smuggle pamphlet 13 back into a series about Japan. Which is either a fitting bookend or exactly the kind of overclaim the whole piece warns about. lol.