THE EVIDENCE LOWER BOUND ·

The tractable floor beneath an intractable log-likelihood. When the evidence log p(x) is out of reach, the ELBO is the number you can actually compute — and every gap between them is exactly one KL divergence wide. Push the floor up, and the ceiling comes into view.

source Jordan, Ghahramani, Jaakkola & Saul — An Introduction to Variational Methods for Graphical Models (1999), Mach. Learn. 37:183–233. Year cited, no stable arXiv id — marked AMBER. Rendered, not quoted.

Blue Team · builds & defends
3
THE MODEL

A constructed linear-Gaussian latent model — chosen so every quantity is closed-form, nothing is trained, nothing is guessed:

z ~ N(0, 1)   x | z ~ N(z, s²)

Then the evidence and the exact posterior are both known in closed form:

log p(x) = log N(x; 0, 1+s²)
p(z|x) = N( x/(1+s²),  s²/(1+s²) )

Because the truth is knowable here, the ELBO can be checked against it to 1e-9 — the point of a laboratory model.

5
THE LINEAGE

This sphere is the-kl-divergence turned into an objective. The KL that measured how far two distributions sit apart becomes the lever you pull:

log p(x) = ELBO(q) + KL( q(z) ∥ p(z|x) )

Since KL ≥ 0, the ELBO is a floor under the evidence; closing the KL raises the floor to the ceiling. Downstream, the-vae maximizes exactly this bound — the reconstruction-minus-prior-KL form is its training loss.

7
THE WITNESS live

Re-runs the bound invariant on a probe q every render. It confirms ELBO(q) ≤ log p(x) — and flips red the instant the KL term is dropped in window 6.

witness idle
The Machine
4
DATA IN in ↓

Fixed model + a variational q(z) = N(m, v) you steer. Everything below is computed live from these.

observed x
1.30
noise s²
0.50
↓
0
THE PANEL booting…

ELBO(q) = Eq[log p(x|z)] − KL(q(z)∥p(z))  =  reconstruction − prior-KL.

reconstruction  Eq[log p(x|z)]
–
− prior-KL  KL(q∥p(z))
–
ELBO(q)
–
log p(x)  (the ceiling)
–
gap  = KL(q∥p(z|x))
–

 

↓
8
DATA OUT out ↓

proven result at boot appears here

bound holds for all q?
–
gap = posterior-KL?
–
tightening is monotone?
–
Red Team · attacks & breaks
1
THE ADVERSARY wall
A mean-field q that factorizes cannot represent a correlated true posterior. The gap KL(q∥p) never closes — the ELBO is a biased estimate of the evidence, not just a noisy one, and no amount of optimization removes that floor-to-ceiling distance.
A tighter ELBO does not certify a better model. Two models are only comparable by ELBO if their bounds are equally tight — and they never are. The bound confounds model fit with the quality of the posterior approximation.
2
THE GRAVEYARD

The ELBO equals the log-evidence.
→ only when q is the exact posterior; otherwise it is strictly below by KL(q∥p(z|x)) > 0.

Maximizing the ELBO maximizes the likelihood.
→ it maximizes a lower bound; a loose q leaves the true likelihood untouched above the slack.

The reconstruction term is the whole objective.
→ drop the prior-KL and the "bound" walks straight through the ceiling — see window 6.

6
THE TAMPER planted void

The disclosed sabotage: drop the KL(q∥p(z)) term and keep only reconstruction. For a sharp q it exceeds log p(x) — no longer a lower bound. The witness in window 7 catches it live.