◄ WORLD V · SONNY 5DART 224 · a helldive at the net

THE METAMORPHIC TEST you can test what you cannot compute the answer to

Often you cannot check a result because you do not know the right answer — the oracle problem. Metamorphic testing sidesteps it: you may not know f(x), but you know a relation between related runs. sin(x) must equal sin(π−x); a search for “cat” must return at least as many hits as “black cat”; sum(xs) must equal sum(shuffle(xs)). Break the relation and you have found a bug — with no oracle at all.

THE TECHNIQUE test a RELATION between runs, not an absolute answer

Check the metamorphic relation sum(v) == sum(permute(v)) — permutation-invariance. A correct sum holds it (difference 0); a position-weighted “sum” violates it, and the test catches the bug with no known answer: live demo







  
HISTORY & CREDIT Chen, Cheung & Yiu, 1998

  
“You cannot test code whose correct output you do not know.” — you can, via relations that must hold between outputs. You never compute the true answer; you check that two related runs agree as they must. A violated relation is a proven bug even when the right answer is unknown. cited

  
1978 · William Howden — coins oracle for a source of correct answers in testing.
1982 · Elaine Weyuker — formalizes the difficulty (“On Testing Non-testable Programs”): for many programs there is no feasible oracle. The phrase oracle problem is popularized later (Barr, Harman, McMinn, Shahbaz & Yoo, 2015 survey).
1998 · T. Y. Chen, S. C. Cheung & S. M. Yiu — metamorphic testing: derive follow-up tests from metamorphic relations, no oracle needed.
now · used to test compilers, ML models, search engines, and scientific code — exactly where oracles are missing.

  
The relation is the oracle you build from symmetry instead of from the answer. Break the symmetry, catch the bug — knowing the truth was never required, only that the truth is consistent with itself. Chen et al. 1998






  
RECOMMEND FOR I-13 relation holds for correct, breaks for buggy, computed

  
On the canonical compiler, the permutation-invariance relation holds exactly for a correct sum (difference 0) and is violated by a position-weighted variant — catching the bug with no oracle:

  
$ i13 run metamorphic.i13   # relation: sum(v) == sum(permutation of v)
good_diff = 0     -- correct sum: permutation-invariant, relation HOLDS
bad_diff  = -2    -- position-weighted 'sum': relation VIOLATED -> bug caught, no oracle

  
Recommend: metamorphic testing is LIT for I-13 — verified the relation sum(v)==sum(perm(v)) holds for a correct sum (diff 0) and is violated by a position-weighted variant (diff ≠ 0), catching the bug without ever computing the “right” total. It is testing by self-consistency — the falsifier for everything an oracle cannot reach.