Value now equals the best you can do this step plus the discounted value of where you land next. One equation, referring to itself — and its unique solution is the best possible plan. Rendered, not quoted.
source Bellman, Dynamic Programming (Princeton Univ. Press, 1957) — archive.org/details/dynamicprogrammi0000bell
A finite MDP: states S={0,1,2}, actions A={a0,a1}, reward R(s,a), transition P(s'|s,a), discount γ=0.9.
The optimality operator T acts on a value vector V:
Because 0≤γ<1 and max is a non-expansion, T is a γ-contraction in the sup-norm. Banach's theorem then hands you a unique fixed point V* — the optimal value function.
Direct child of the-markov-decision-process: the MDP supplies the (S, A, R, P, γ) tuple; the Bellman equation is its recursive optimality condition — the self-referential equation V* = T V* whose solution is the best possible plan.
Downstream, this same operator is what value iteration, policy iteration and Q-learning are all quietly iterating toward.
Live re-check. Recomputes the residual ‖T V* − V*‖∞ against the boot-sealed V* using the current γ. Untouched: green. If the Tamper (6) flips γ>1, the fixed point breaks and this badge goes red.
A value estimate V = [V(0), V(1), V(2)] — your current guess of each state's worth. Edit it, then push it through T.
Apply T once to your V, or iterate T to its fixed point. All numbers below are computed live from R, P, γ — nothing is baked.
The proven result: the fixed point V* and the greedy policy that attains it, computed at boot and gated by selfcheck().
The contraction guarantee is load-bearing on γ<1. Let γ≥1 and T is no longer a contraction: value iteration can diverge, the fixed point is not unique or not attracting, and "value now = best now + discounted next" loses its unique solution. Undiscounted (γ=1) infinite-horizon problems need extra structure (proper policies, average-reward theory) — the plain equation does not close.
The disclosed planted void. Sets γ = 1.6 > 1, turning T into an expansion. The boot V* stops being a fixed point — and the Witness (7) catches it live.