test: the run counts itself, and the floor is one per suite - #450
Merged
Conversation
The summary's figures were restated beside the suites rather than counted off them, and the substantive floor was a hand-maintained sum. Both drifted, and the drift was silent because a floor below the truth still fires. Wrapping the calls put a number on it for the first time: 18 of the 47 hand-written increments were wrong, the summed floor stood at 435 against a run of 551, and one suite had no increment at all, so its four cases had never counted toward the floor in any run. The run now brackets each suite call and counts the case lines that suite printed. The summary asks the tally for its figures, the floor is one per suite rather than one on their sum, and a suite that reports it ran and then measures nothing exits 2 by name — even when the total grew around it. Adding a control to a wrapped suite now needs no summary edit at all, measured rather than asserted: planting one extra case moved the printed figure and the run stayed green with no edit to the summary. 34 of the 44 clause figures are on the tally. TY01 is derived rather than tabled — it captures what reportCase actually prints and holds the scanner against it two-sidedly — and TY02 holds the scanner against every gutter word this run printed, because a scanner that stops matching goes silent rather than red. Five mutants were planted in the real file and run: a suite emptied, a suite call unwrapped, the case gutter reworded, a summary naming a suite that did not run, and one wrapper over two suites. Four figures the tally derives disagree with what the summary states, so they are left as literals with both numbers recorded beside them. Three are a definition disagreement rather than staleness and need a ruling; the fourth is stale under every reading. Filed separately. Refs #439 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Refs #439. Landed overnight;
bun run typecheckclean,bun run lintclean,bun run selftest559 PASS / 0 FAIL / 0 SKIP / 0 HOLE.What the drift actually was
The issue reported one stale count (a suite's floor at 24 against 29 cases). Wrapping the calls put a number on the whole surface for the first time:
runLoopSeamSuite— its four cases had never counted toward the floor in any run)Worst rows:
path-slider25/47,contour-mesh8/30,check3/19,packer18/25,chainfit12/18.The mechanism
console.logis wrapped for the duration of the suite phase; each suite call is bracketed and the case lines it printed are counted (PASS/FAILat the two-space gutter;SKIP/INFOare seen and deliberately not counted; anything else faults). The summary asks the tally.tallyFaults()runs before the summary reads a single number off it.🔒 The floor is one per suite, never on their sum.
TY04is the clause that makes that matter: a set whose total (20) is higher than the healthy set's (5) still faults, because one suite is dead.⭐ Adding a control to a wrapped suite now needs no summary edit at all — measured, not asserted: planting one extra case moved the printed figure
34 → 35with zero edits and the run stayed green. 34 of the 44 clause figures are on the tally. That is what #440/#441/#443/#444 were queued behind.Gate —
TY01–TY08TY01is derived rather than tabled: it captures whatreportCaseactually prints and holds the scanner against it in both directions.TY02holds the scanner against every gutter word this run printed — because a scanner that stops matching goes silent, not red.Five mutants planted in the real file, run, and restored:
the suite "…" reported that it ran and then printed no PASS or FAIL line at all4 case line(s) were printed outside every tallied suitereportCaseprintsOKAYTY01/TY02redno suite ran under the key "…"the suite "…" opened 2 section header(s)Verified independently before merge: emptying
runSlotSuiteon this branch exits 2, naming it twice — "reported that it ran and then printed no PASS or FAIL line at all" and "opened 0 section header(s); a suite opens exactly one".selftest.tsrestored and checksum-matched.Also measured: corpus parked → 508 PASS / 5 SKIP / 5 HOLE, exit 0 (the "did not run" branches behave);
--cutsnamed-and-missing exits 2 and present-but-broken exits 1, unchanged.🔒 The printed summary is byte-identical to
main@0cd287b's apart from the one inserted clause for the new controls, and all 34 interpolated figures reproduce the hand-written number exactly.The STOP condition fired, and nothing was edited around it
Four figures the tally derives disagree with what the summary states. They are left as literals, each marked in place with both numbers recorded beside them, and filed separately for a ruling — because three of them are a definition disagreement rather than staleness (some suites count their own positive control in their figure and some do not), and no single derivation reproduces the text. The fourth is stale under every reading.
Two more of the same family, reported not fixed: one suite of ten controls is described in the summary nowhere at all, and one corpus-dependent suite announces its absence nowhere in the section list.
Rejected approaches
CUR08's pattern) — it keeps the hand-written number, so adding a control still costs a summary edit, which is the exact cost the four queued cards are waiting to be rid of.SKIPas substantive — rejected on measurement: all sixSKIPsites mean "this did not run", so counting them would let a corpus-less run book five holes as coverage.HOLEin the gutter vocabulary — measured:HOLEonly ever prints in the ten-space detail gutter, so it would have been a branch no control can reach.Known limitation, stated plainly
This removes the stale count rather than gating the text. Nothing reads the printed summary, so a deliberate re-hardcode would not be caught. That is the price of the source scanner rejected above, and it is what a later card could still add once the four deltas are ruled on and the literals disappear.