Skip to content

fix(scripts): doc-authoring self-test pins its batteries by NAME, so a battery that stops running is named rather than uncounted - #13487

Merged
os-project-manager merged 2 commits into
mainfrom
claude/issue-13173-doc-authoring-self-test-floor
Aug 30, 2026
Merged

fix(scripts): doc-authoring self-test pins its batteries by NAME, so a battery that stops running is named rather than uncounted#13487
os-project-manager merged 2 commits into
mainfrom
claude/issue-13173-doc-authoring-self-test-floor

Conversation

@os-project-manager

@os-project-manager os-project-manager commented Aug 30, 2026

Copy link
Copy Markdown
Collaborator

Fixes #13173

selfTest() in scripts/check-doc-authoring.mjs decided success as
failures.length === 0 with no assertion-count floor anywhere, so the success
condition and the total-failure condition produced identical output — in the
gate that exists to catch prose claiming more than the code delivers.

The mechanism, re-measured on current main (not quoted from the card)

The file changed three times the day the card was filed, so the experiment was
re-run rather than cited. On 8fcd0816, an early return injected at the top
of selfTestRule3():

MUTATION injected_marker_count=1 post_blob=f3bf0c75 head_blob=864bd697
ABLATED_SELFTEST_EXIT=0
✓ check-doc-authoring self-test: scope wiring (...), the dead-root hard error (...
RESTORE hash-object=864bd697 HEAD_BLOB=864bd697   (byte-identical)

Exit 0, with the full success line asserting those cases hold. Premise confirmed.

The audit, done before anything was designed

The card recorded that no audit of the current registrations had ever been made.
One was made here, by instrumenting expect to log its call site and diffing
every executed call site against every static one:

batteries registering assertions 16
assertions executed 198 (selfTest 72 · selfTestRule3 108 · selfTestPackagesProse 18)
static expect() call sites 178
static call sites that never executed 0

The one static hit that does not execute is expect(issue.message).toContain(...)
inside a fixture string — text, not a call site.

So the PM hypothesis that at least one battery was already dark is FALSIFIED:
every battery runs today.
That is what let the floor be pinned to today's real
counts rather than to a guess.

The repair: registered NAMES, per battery

Not a count. A count answers "something moved"; the card's own 40-to-3 warning
needs "which battery moved", and a pinned TOTAL rots the moment a sibling
grows. So this takes the shape the sibling check-dispatcher-error-vocabulary
gate landed hours earlier — the registered names, held as a set:

  • SELF_TEST_BATTERIES declares 16 batteries, each with its own case floor.
  • Each battery opens with battery('...'); every expect() is attributed to the
    battery most recently opened. An assertion with no battery open is filed under
    a key that is not declared, so it reds by the same set difference instead of
    inflating whichever battery ran last.
  • The floor requires the opened set to equal the declared set, with each
    battery at or above its own count, and it is evaluated before the verdict —
    so that line can only be printed by a run in which every battery ran.
  • The registry's own size is pinned too: deleting an entry silences its floor
    exactly as effectively as zeroing it.
  • main() no longer does return selfTest(). selfTest() returns a verdict
    sentinel only after the floor has been evaluated and the success line printed,
    so a return that leaves the function early is refused rather than exiting 0
    with no output at all. Same defect class, one level up; named here because it
    is beyond the literal "assertion-count floor" the card asked for.

Counts are a floor, not an equality: adding cases must never red. Falling
below one means cases stopped running — the battery is the bug, and the failure
text says so, with lowering a floor marked ⛔ MAINTAINER-ONLY beside the
RATCHET_AUTHORITY_MARKER this file already carries.

The ablations — the floor NAMES the battery, five ways

Every mutation proven on disk by git hash-object against the HEAD blob (never
by an editor's exit code), every restore proven byte-identical with
git diff HEAD empty.

# mutation exit the floor says
A early return at the top of selfTestRule3() 1 names all 7 of its batteries, each DID NOT RUN — 0 cases registered, N pinned
B one precision case deleted (15 to 14) 1 battery "published-catalog id precision" registered 14 case(s), below its pinned floor of 15
B2 the card's 40-to-3 case: mustPass cut from 12 entries to 1 1 registered 4 case(s), below its pinned floor of 15
C early return at the top of selfTest() itself 1 selfTest() returned without reaching its verdict
D a battery entry deleted from the registry 1 declares 15 batteries, below the pinned 16 and battery "spec text: precision" registered 18 case(s) but is not declared

Before this PR every one of those exits 0 with the full success line.

Verification

Both legs, on the final commit 30cae64d — the CI step is --self-test && the
main run, so both are quoted:

$ node scripts/check-doc-authoring.mjs --self-test          # exit 0
✓ check-doc-authoring self-test: ... all hold. — 16 declared batteries,
  198 cases registered, every battery at or above its pinned floor.

$ node scripts/check-doc-authoring.mjs                      # exit 0
✓ doc authoring guard: 393 files clean — no bare metadata literals.
✓ doc authoring guard: 48 published skill files clean — no internal issue-id references.
✓ doc authoring guard: 14083 customer-facing string(s) across 691 spec sources clean ...
✓ doc authoring guard: sibling-package prose ids hold the baseline — 831 pinned site(s) ...

Gate family derived from the real changeset with
node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack (13
path-matched + 2 convention-triggered for editing a gate script), all run on
30cae64d, all exit 0:

check:doc-authoring · check:agent-test-spelling · check:bash32-floor ·
check:cli-command-ids · check:cross-package-test-inputs · check:entry-guard ·
check:parse-guard · check:pnpm-filter-targets · check:watch-hint-literal ·
check-ci-filter-parity · check-cross-package-test-inputs ·
check-shard-attestation · pm/bare-root-worklist --self-test ·
check:pm-dispatch-gates

Added beyond the derivation because this diff plainly implicates them, both
exit 0:

  • check:nul-bytesOK (scanned 7451 text file(s) ... no raw ASCII control bytes),
    plus a direct grep -naP over the edited file: no control bytes.
  • check:ratchet-remedy-authority — this file is classified marked in that
    gate's control corpus, and the new refusal text is a second marked offer:
    OK check-ratchet-remedy-authority: 170 scripts swept; 10 mark the expanding remedy ⛔ MAINTAINER-ONLY ... Control corpus: 24 hand-classified scripts, set-equality audited both ways.

check-test-completeness.mjs exits 3 PREREQUISITE NOT MET locally by design
(it grades a saved turbo run test log). Recorded NOT MEASURED — neither
red nor green.

No test suite in the repo pins this script (git grep over *.test.ts /
*.spec.ts for check-doc-authoring: zero hits), so no pin suite is owed.

Lint, narrowed with the narrowing proved rather than run repo-wide:

  1. Universe read from ESLint's own config resolution, not from a guess about
    what counts — the file resolves to a real config and is linted, not ignored.
  2. File count read from --format json: 1 file, 0 errors, 0 warnings.
  3. Invariance for untouched files: this repo runs one eslint.config.mjs which
    never enables type-aware linting for any file (no parserOptions.project,
    no typed @typescript-eslint rules) — stated in that config's own header
    with a measured positive control. A change confined to one file therefore
    cannot move any untouched file's verdict.

CI runs the full farm regardless; 166 of the 187 discovered families sit behind
no path filter at all.

Not in scope

Whether other --self-test implementations in scripts/ share this shape is
still an open survey, deliberately untouched here — that is the class card's
question, not this one's, and this PR does not address it.

No changeset: scripts/-only, publishing nothing from any package, matching the
two most recent PRs to this same file, which both carried skip-changeset and
no changeset file.

Authored by Claude Code, dispatch session session_01Pk26oZ12t5N1hwGW1m1MgC
(https://claude.ai/code/session_01Pk26oZ12t5N1hwGW1m1MgC) — recorded in prose
because a body edit rewrites the session-URL footer to its bare form.


Generated by Claude Code

claude added 2 commits August 30, 2026 14:29
… by a bare pass

`selfTest()` decided success as `failures.length === 0` with no assertion-count
floor anywhere, so "every case held" and "the cases never ran" produced the same
output: an early return at the top of `selfTestRule3()` left that whole battery
unrun and the script still exited 0, printing the verdict line asserting those
cases hold — in the gate that exists to catch prose claiming more than the code
delivers.

Every battery now declares itself with `battery('<name>')`, every `expect()` is
attributed to the battery most recently opened, and the floor requires the
OPENED set to equal the DECLARED set with each battery at or above its own
pinned count. A set difference names WHICH battery stopped running; a count says
only that something did. A bare non-zero total would not have caught the 40->3
case the card warns about, and a pinned TOTAL rots as soon as a sibling grows.

`main()` no longer does `return selfTest()`: `selfTest()` returns a verdict
sentinel only after the floor has been evaluated and the success line printed,
so a `return` that leaves the function early is refused instead of exiting 0
with no output at all.

Audited before designing: 16 batteries, 198 assertions, and every static
`expect()` call site in the file executes today — no battery was already dark.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pk26oZ12t5N1hwGW1m1MgC
…ery's count

Deleting an entry from SELF_TEST_BATTERIES silences that battery's floor exactly
as effectively as zeroing its count, so the registry size is pinned too — the
same shrink-only shape, with the same MAINTAINER-ONLY refusal beside it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pk26oZ12t5N1hwGW1m1MgC
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/m skip-changeset PR has no user-facing published change; bypasses the changeset gate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[finding] check-doc-authoring.mjs --self-test exits 0 while registering ZERO cases — and prints its full success line asserting they all hold

2 participants