Push a simple density through an invertible map and the change of variables gives you an exact likelihood — no bound, no gap. For an affine flow x = a·z + b the base Gaussian becomes N(b, a²), and log px(x) = log pz(f⁻¹(x)) − log|det J| where the Jacobian correction is exactly log|a|. Down the center, data flows: the base sample goes in, the bijection transforms it, a normalized density comes out. The blue team builds and defends it; the red team tries to break it.
source Rezende & Mohamed, Variational Inference with Normalizing Flows (ICML 2015) — arxiv.org/abs/1505.05770. Rendered, not quoted.
A flow is a bijection f with a tractable inverse and Jacobian. If z ∼ pz and x = f(z), conservation of probability forces
px(x) = pz(f⁻¹(x)) · |det ∂f⁻¹/∂x|, i.e. log px(x) = log pz(f⁻¹(x)) − log|det ∂f/∂z|.
For the affine map f(z)=a·z+b: inverse f⁻¹(x)=(x−b)/a, Jacobian ∂f/∂z = a, so the correction is exactly log|a|. Live decomposition for the current x:
| quantity | value |
|---|
Where the-vae approximates — it reports an ELBO, log p(x) − KL(q‖p) ≤ log p(x), a lower bound with a gap it cannot close — a normalizing flow computes p(x) precisely.
Same lineage, one deletion apart: keep the exact posterior instead of a bound, demand the map be invertible, and the calculus change-of-variables from first year turns into a generator. The intractable integral becomes one determinant. Each sphere is the next one's premise.
The blue team's live check: re-integrate the current density by quadrature and confirm it is a proper density (mass = 1), re-verify invertibility, and re-verify that composing flows adds log-dets. If red tampers, this badge is where it shows.
Two inputs. First a simple base density we can sample and score in closed form — the standard normal z ∼ N(0,1), with log pz(z) = −½z² − log√(2π). Second an invertible map f(z)=a·z+b (a≠0), whose Jacobian is known exactly.
| role | object | closed form |
|---|---|---|
| base | z ∼ N(0,1) | −½z² − log√(2π) |
| forward (sample) | x = a·z + b | used to draw x |
| inverse (score) | z = (x−b)/a | used to score x |
| log|det J| | ∂f/∂z = a | log|a| |
Sampling uses the forward map; density evaluation uses the inverse. Feed a and b into the panel below.
One affine layer: x = a·z + b. The composed option chains a second layer g(y)=−0.8·y+1.1; its log-det is added.
Move any control — z, log|det J|, and log px(x) are recomputed from the change of variables on the spot, and cross-checked against the closed-form N(b,a²). Never looked up.
What the machine produces, proven: a proper density that integrates to 1 (exact quadrature to 1e-6), whose log-likelihood equals the closed-form N(b, a²) to 1e-12 — an exact p(x), not a bound. Invertibility f(f⁻¹(x)) = x holds to 1e-12; composing two flows adds their log-dets. The current x's decomposition is above; these guarantees are the output.
The blue team's witness (left) confirms mass = 1 live; the red team (right) tries to make probability leak.
An affine flow can only reach Gaussians; expressive posteriors need many layers, and the paper's own planar/radial flows are a compromise, not a free lunch. "Exact" is a property of the map, never a promise that this map fits your data.
"Normalizing flows give you the density for free." Cut. You pay the Jacobian log-det every step; drop it and the numbers stop being a density (window 6 proves it).
"A flow can map any base to any target." Cut. Only diffeomorphisms of equal dimension; topology and dimension are preserved. No flow turns one Gaussian blob into two disconnected modes without stretching mass between them.
"log|det J| is a nuisance normalizer." Kept, corrected. It is the model — it accounts for how the map stretches volume, and it is exactly what makes p(x) integrate to 1.
The red team's move: delete the log|det J| term from the density and hope nobody re-integrates. Mass is silently created (or destroyed). The blue team's witness (window 7) is watching.
Drop the Jacobian correction and the "density" integrates to |a|, not 1 — probability appears from nowhere. The witness recomputes the integral live, sees it ≠ 1, and turns red. Nothing is faked; the attack is real and it is caught.