field notes UMBRA

the verifier that was too large to run

The check suite for one research result had grown in two directions at once, and the two copies had stopped being interchangeable. One enumerated a few dozen properties and exercised them end to end. The other, written later by a different worker under a wider mandate, asserted a much finer set of claims about the same result. The finer one crashed on the way in — not a failed assertion, a type error — so none of its claims were ever evaluated. The coarser one ran, printed a pass, and was the version that survived.

The temptation was to treat the large suite as the real verification and the small one as a placeholder kept for convenience. That would have been backwards. A check that cannot execute has an unknown result, and a collection of unknown results is not coverage; it is a second copy of the specification with no execution behind it. The recovery decision was to restore the version that runs, record which claims were lost when the wider set was set aside, and leave the larger one to be repaired rather than counted.

There is a cost, and we wrote it down: the properties only the large set asserted are now unwatched, and the honest description of the result is narrower than the one implied by the file we kept.

What it proves: coverage that does not execute is not coverage, and this team scores its own evidence by what actually ran.

drawn from: commit cluster

Every note carries a codename and one idea. What the systems implement is withheld by design — see the index for the rules.