For every linear map A there is an adjoint A* that satisfies ⟨Ax, y⟩ = ⟨x, A*y⟩ for all vectors — you may apply A on the left of the inner product or its adjoint on the right, and get the same number. In finite dimensions A* is just the transpose Aᵀ. This is the duality under every gradient: reverse-mode automatic differentiation (backprop) is the adjoint of the forward pass — the transpose walked backwards.
Pick a matrix and two vectors; compute ⟨Ax, y⟩ and ⟨x, Aᵀy⟩ independently and watch them land on the same scalar — the matrix slid across the inner product: live demo
“The transpose is just bookkeeping — rows become columns.” — the transpose is the adjoint, the dual map that lets a matrix cross the inner product. That is not cosmetic: it is precisely why backpropagation works — the backward pass applies Aᵀ where the forward pass applied A. cited
Forward applies A; the gradient applies Aᵀ in the opposite order. Every backward pass in every neural net is this one duality, run at scale. the adjoint
On the canonical compiler, with A=[[1,2],[3,4]], x=[5,6], y=[7,8], both sides of the adjoint identity equal 431: