The honest curvature test. To ask whether attention has a real geometry, convert the attention graph to geodesic distances (shortest paths where strong attention = short edges — this is Isomap), then run classical MDS: double-center the squared-distance matrix and eigendecompose. Positive eigenvalues are Euclidean dimensions; negative eigenvalues mean the distances literally cannot be laid flat in Euclidean space — a true signature of intrinsic (hyperbolic) curvature, not a drawing choice. The 2D bars are that spectrum, live.
What the three modes show. Tree (hub + sub-hubs): ~6 intrinsic dimensions, a few percent negative mass — real but mild hyperbolic curvature from the hierarchy. Grid (pure recency): collapses to ~2 dimensions — the tokens form essentially a 1D chain, the sequence as a line, which you can see directly in the 3D embedding. Shuffled: ~22 dimensions, zero curvature — high-dimensional noise with no manifold at all. The pipeline cleanly separates structure from noise, which is the useful part.
The verdict on "useful or just looks good." Both, but honestly weighted. The relational embedding is genuinely useful — it recovers the intrinsic dimensionality and separates a chain from a hierarchy from noise. But the curvature specifically is real and modest — a few percent, a slight hyperbolic tilt when structure is present, none when it isn't. It is not the dramatic gravity-well the shape might suggest; that resemblance is the seductive-but-misleading part. The rubber-sheet gravity picture lies by bending a 2D sheet into a fake third dimension; this instrument refuses that move — the negative-eigenvalue mass is the only curvature claim it makes, and it's small. Amber tier: the attention is modeled, but the MDS, geodesics, and eigendecomposition are exact, computed live in the browser on every mode switch.