Purple Paper · side-sheet · learning machines · II

The Crossbar — where the dot product becomes a current

Set a weight as a physical conductance. Drive the input in as a voltage. Ohm's law does the multiply; Kirchhoff's law does the sum — on a bare wire, for free. The atom from the last paper stops being computed and becomes a measured quantity. Below it runs live, then we walk straight into the gates that keep it honest.
Chua 1971 postulates the memristor  →  HP 2008 builds one  →  conductance = weight, voltage = input, I = V·G = a multiply by physics  →  a grid of them = one matrix-vector multiply in a single analog step.

I · One device — the multiply, by Ohm's law

A memristor is a resistor that remembers: you set its conductance G and it stays. Drive a voltage V across it and the current that flows is I = V·G — input times weight, happening in the material, instantly, with no multiplier circuit anywhere. Drag both and watch the product.

one memristor — voltage in on the left rail, current out on the right; line brightness ∝ current
I = V · G = +0.80 · 0.65 = +0.52
Negative V just reverses the current. A real device also can't make G negative — conductance is a physical quantity, always ≥ 0. Hold that thought; it's the reason signed weights need a trick in Module III.

II · One column — the sum, by Kirchhoff's law

Now wire several memristors to one shared output wire. Each pushes its own current Vᵢ·Gᵢ onto the rail, and Kirchhoff's current law says currents on a shared node simply add. The wire carries Σ Vᵢ·Gᵢthat is the dot product, performed by physics, as a single measurable current. Drag the inputs; the rail sums them.

four memristors feeding one rail — each contributes Vᵢ·Gᵢ, the rail carries the sum at the bottom

III · The crossbar — a whole layer in one step

Lay the devices in a grid. Input voltages enter along the rows; every column sums its own dot product onto its bit-line. One set of inputs, and all columns answer at once — that's a full matrix-vector multiply, an entire layer of perceptrons, computed in memory where the weights physically sit. Below: a 25→4 crossbar storing four bipolar templates. Draw a pattern; the columns score it; the brightest column is the recognized shape.

draw on the pad →
input pad · 5×5 → 25 voltages
click a cell to toggle it on (+1) / off (−1)
25 word-lines (rows) × 4 bit-lines (columns). violet crosspoint = stored +1, rose = stored −1. a crosspoint lights when input and weight agree; current flows down each column.
column currents = match scores
recognized
effective bits
With clean inputs the four templates sit far apart, so even heavy device noise rarely flips the winner — that robustness is exactly why analog crossbars target inference. Flip a few pixels to shrink the margin, then raise σ, and watch the scores wobble until the wrong column wins.
The signed-weight trick. A conductance can't be negative, but a weight can. So each weight is stored as two devices, G⁺ and G⁻, and the column reads the difference: w = G⁺ − G⁻. Wire two physical rails, subtract their currents, and you've recovered the full signed range — at the cost of doubling your device count. (Verified: the differential pair reproduces the signed dot product to exactly zero error.)

IV · The gates — why it isn't a free lunch

The physics is real and the efficiency pitch is real — but it's a trade, not a miracle. Four things keep it honest, and the numbers below are the ones that came out of the bench-check.

where the energy actually goes (illustrative budget for an analog MVM tile)
MAC ~8% DAC ~14% ADC ~62% periphery ~16%
the dot product is nearly free; reading the analog answer back out (ADC) eats most of the win. this is the field's standing embarrassment, not a detail.

① analog noise sets the ceiling

Conductance has device-to-device spread and cycle-to-cycle drift. Your weight isn't 0.4417, it's 0.44 ± a smudge. Measured: ~2% per-device variation already caps you near 4.6 effective bits; 5% → ~3.3 bits; 10% → ~2.3 bits. There is no float32 here.

② the ADC tax

Inputs arrive digital, the answer comes out as an analog current, so every column needs an ADC — and ADCs are large and power-hungry enough to eat most of the theoretical efficiency win (the bar above). The MAC being free doesn't help if reading it isn't.

③ writing is slow & wears out

Setting a conductance precisely is slow, imprecise, and degrades the device over many writes. Great for fixed inference weights you write once; painful for training, which demands constant precise updates. This is why crossbars do forward passes, not backprop.

④ sign costs you double

Conductance is positive-only, so every signed weight is a G⁺ − G⁻ pair — two devices per weight. The math is exact (Module III), but your array is twice the size it looks.

The honest one-liner. You trade digital precision and flexibility for analog speed and energy, and you only come out ahead on noise-tolerant, fixed-weight, inference-heavy work. The argmax of a well-separated classifier survives all of this (still ~96% correct at σ=50% in the bench-check); a task that needs the actual value to many bits does not.

V · The prize, and where it doesn't belong

The reason any of this is worth the trouble is the memory wall. In a GPU running an LLM the dot products are cheap; what costs the energy and the time is hauling billions of weights out of memory to the compute units, every step. The crossbar computes at the weights — they never move — which is the whole pitch: potentially orders-of-magnitude better inference efficiency, for the right workload.

the bottleneck

von Neumann's wall

Weights live in memory; compute lives elsewhere; the wire between them is the cost. Most of an LLM's inference energy is data movement, not arithmetic.

the move

compute-in-memory

The crossbar collapses memory and compute into one device. The weights don't travel to the math — the math happens where the weights already are.

the limit

not for branches

A branch predictor needs a deterministic digital bit in ~0.25 ns. An analog MVM plus ADC at that timescale is a non-starter — same timescale wall that killed "is it networked." Crossbars are for big fixed weight matrices, not one-bit guesses.

The curse, nearly broken. The memristor is genuinely recent — 1971 in theory, 2008 in silicon — one of the rare pieces in this whole chain that isn't two centuries old. But it doesn't quite let you go: using networks of tunable conductances to compute weighted sums is exactly what mid-century analog computers did with resistor banks. New device (~50 years), old idea (~70+). You half-escaped this time. lol.