Loss falls as a power law in model size. Feed a noiseless synthetic loss curve L(N)=L∞+(Nc/N)^α, subtract the known floor, and the exponent α drops out of an exact closed-form log–log regression — no training, no iteration. Bigger, reliably, is better; this machine says by how much, and proves it.
SOURCE Kaplan et al., Scaling Laws for Neural Language Models (2020) — arxiv.org/abs/2001.08361. Rendered, not quoted.
A single mechanism generates the curve: L(N)=L∞+(Nc/N)^α. The floor L∞ is the irreducible loss; the term (Nc/N)^α is the reducible part that shrinks with size.
Subtract the known floor, take logs of both sides: log(L−L∞) = α·log Nc − α·log N. A straight line. Its slope is exactly −α.
Fitting the exponent is therefore ordinary least squares — one matrix solve, closed form, no gradient descent.
Neighbour: the-power-law. That sphere holds the bare mathematical object — a quantity that scales as x^k, straight in log–log. Here it is measured on the loss curve: the empirical rule that turns "bigger is better" into a number.
The-power-law gives the shape; THE SCALING LAW recovers its slope from data and defends that the recovery is exact.
Live re-check of the panel’s fit against the truth: does the recovered α still match, is the log–log fit still perfectly linear? It re-runs on every tamper.
Seven noiseless points from a true power law: N = 1e6 … 1e9, geometric spacing, with true α=0.076, L∞=1.69, Nc=8.8e13. One point at N=1e10 is held out of the fit.
Proven result: on a true power law the log–log OLS slope recovers the exponent to machine precision (≈1e−17), the fit is perfectly linear (R²=1), and it extrapolates a held-out size exactly. The scaling exponent is not estimated — it is read off.
The power law is a description of a regime, not a promise. Kaplan’s law holds across the measured range; it does not license infinite extrapolation. Curves bend: an irreducible floor L∞ exists, and later work (Chinchilla) re-weighted data vs. size — the compute-optimal allocation shifted. The clean exponent here is exact only because the data is a power law; real loss curves are noisy and only approximately straight in log–log.
"Bigger models always beat smaller ones."
→ Only at matched training; a huge under-trained model can lose. Scaling laws assume near-optimal training at each size.
"Fit L vs N directly and read the slope."
→ L vs N is not a line; a linear fit there returns a wrong exponent and R²<1. You must fit log(L−L∞) vs log N. (See window 6.)
"Ignore L∞ and just fit L."
→ The floor bends the log–log line; subtract the known L∞ first, else the recovered α drifts.
Disclosed planted void: force the fit into linear space (L vs N, not log vs log). The recovered exponent goes wrong and the R²=1 linearity check breaks. The Witness (7) catches it live.
the-scaling-law. The engine is genuinely LIT: it computes the exponent live by exact closed-form regression, a fail-loud selfcheck() throws on any violation at boot, and the planted void in window 6 is caught by the live witness. Built by David Lee Wise (ROOT0) with AVAN.