Skip to content

fix(devx): anchor the self-test floor probe on the real definition, not the first text that reads like one - #15580

Merged
baozhoutao merged 3 commits into
mainfrom
claude/issue-14963-probe-anchor-real-definition
Sep 4, 2026
Merged

fix(devx): anchor the self-test floor probe on the real definition, not the first text that reads like one#15580
baozhoutao merged 3 commits into
mainfrom
claude/issue-14963-probe-anchor-real-definition

Conversation

@claude

@claude claude Bot commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Fixes #14963

injectEarlyReturn took the FIRST match of function NAME( against RAW
source. In scripts/pm/dispatch-gates.mjs a docblock sentence and then a
FIXTURE STRING stand ahead of the real definition, so the ledger carried

// Injecting into this file produces a SyntaxError (the anchor lands inside a
// template literal), so no run of it measures anything.
'scripts/pm/dispatch-gates.mjs': null,

— a limit of the INSTRUMENT recorded as a property of the FILE, and read that
way for months.

⚠️ The card calls that row NOT MEASURED. What the defect actually produces is
worse and is now written down: a copy that dies in the parser exits non-zero
and prints a stack, so mutatedSpoke is true and the three-valued verdict
scores it HELD — a hold awarded to a gate for the probe having broken its
own copy of it. Only the hand-written null kept that off the sweep.

The repair — TWO rules, neither of them redundant

rule what it is
MASKED the match is taken over maskCommentsAndLiterals(src)scanSource's existing comment and literal flags OR-ed and handed to the module's own blank(). Offsets and line numbers survive, so the index found in masked text slices the ORIGINAL.
LINE-START the match must BEGIN a line; export / export default / async are the only prefixes a definition in this tree carries.

scripts/js-comment-mask.mjs is not touched. Its origin/main state already
exports everything this needs (scanSource returns { comment, literal, interpolation } and flags interpolation bytes as literal at the end of its
pass; blank takes a flag array). Nothing was missing, so #15532 is left to
extend that module on its own.

The POPULATION criterion keeps reading maskComments. Every --self-test
dispatch names the flag with a string literal, so the code-only mask would not
classify a file generously — it would empty the census, taking this
instrument's own control fixtures (deliberately strings) with it. A control now
pins the two masks to their OPPOSITE answers on the same fixture.

The controls, inline on every invocation, in the file's convention

(1) the anchor — one fixture carrying three decoys ahead of its real
definition, one per way the anchor can take the wrong text, in the order they
occur in dispatch-gates.mjs: a docblock sentence (COMMENT), a fixture the gate
feeds its own scanner (TEMPLATE LITERAL), and a named function expression held
as a value (real CODE, MID-LINE). Six fixture-validity assertions pin that each
decoy is the kind it claims to be (scanSource flags read directly), that an
unmasked first match lands on a decoy, and that masking ALONE still lands on the
mid-line one — so neither rule can be carrying the other. Then the injection
position, and the fixture is SPAWNED: the real definition is holed, so the one
reading that can only come from anchoring on it is DEFEATED with zero bytes
printed — which also says the copy parsed and ran.

(2) the two masks stay apartDISPATCH.test(maskComments(HOLED_GATE))
holds (unchanged) and DISPATCH.test(maskCommentsAndLiterals(HOLED_GATE)) must
be false. Census count unchanged: 178, and the whole non-probe census
output is byte-identical to main.

(3) the existing controls — sentinel HELD, helper-handshake HELD /
DEFEATED on the one-line pair, ACCIDENT, and the baseline precondition — all
still read what they read, unmodified. The three-valued verdict, the #15455
precondition, the #15491 helper control and the #15517 DISPATCH regex are
untouched.

The ledger row

'scripts/pm/dispatch-gates.mjs': null becomes 'scripts/pm/dispatch-gates.mjs': 'selfTest'. It stays hand-read — raw source holds four self-test-shaped names —
but for THIS FILE's reason, not the instrument's, and the comment now says the
measured thing.

The check-platform-checklist.mjs row stays as a null; only its prose is
corrected. The card says FIVE self-test functions and the comment said FOUR;
both are stale — the dispatch calls SIX (selfTestTrapVocabulary,
ProvisioningUse, UnreferencedRecipes, MetaCallSpelling,
SourceLineCitations, SymbolAnchors, each with its own
requireReachedVerdict on both legs). The same "four" in the
HELPER_HANDSHAKE_GATE docblock is corrected too, and the ledger docblock's
"nine files" is ten rows.

The dispatch-gates probe reading — MEASURED, and it is still NOT MEASURED

Narrowed to that one row through main()'s own selection (ENTRY_BY_HAND
first). A full --probe runs 178 self-tests twice and does not fit this
container's ~10-minute foreground cap: NOT MEASURED, said as such.

ENTRY_BY_HAND row: "selfTest"   mechanical defs: ["selfTest","selfTestOnlyCallables","maskSelfTests","fixtureSelfTest"]

--- main()'s default timeout (120s) --- 120.2s
{ "verdict": "NOT MEASURED", "why": "killed by SIGTERM" }

--- extended timeout (900s) --- 399.0s
{ "verdict": "NOT MEASURED", "why": "baseline run failed (exit 1)",
  "entry": "selfTest", "baselineExit": 1, "baselineBytes": 141759,
  "baselineHead": "  ✓ extracts plain pnpm check" }

Two different limits, each masking the other, and neither is this card's:

cp scripts/pm/dispatch-gates.mjs scripts/pm/.self-test-floor-probe-dispatch-gates.mjs
node scripts/pm/dispatch-gates.mjs --self-test          # EXIT=1
  ✗ the tier constant's VALUE is spelled in exactly ONE site under
    .claude/skills/pm-dispatch + scripts/pm — 22+18 files read, the definition at
    scripts/pm/dispatch-gates.mjs:9020 the only one allowed
    (found: scripts/pm/dispatch-gates.mjs:9020,
            scripts/pm/.self-test-floor-probe-dispatch-gates.mjs:9020)
  ✗ dispatch-gates self-test: 1 of 1415 case(s) failed.

⛔ Not folded in, and not worked around. #15515 is a separate queued card on this
same file.

What this card's repair does deliver, measured by hand the way #14960 did —
the anchor now lands on the definition and the copy is RUNNABLE, which is
exactly what the null row denied:

anchor byte 624612, line 11507 -> "function selfTest() {"
marker count on disk: 1
node -c .self-test-floor-probe-dispatch-gates.mjs                 EXIT=0   (it PARSES)
node .self-test-floor-probe-dispatch-gates.mjs --self-test        EXIT=1   202 bytes
  ✗ dispatch-gates self-test: selfTest() returned without reaching its verdict,
  so no success line was printed. Exiting 0 here would report a self-test
  that never finished as a self-test that passed.

That is #14960's expectation reproduced on today's 1415-case count: before the
handshake landed, an early return took this gate from 1288 cases pass to zero
bytes and exit 0; after it, exit 1 with the named diagnostic. The probe would
score this HELD the moment #15573 and #15515 are out of its way.

Verification — every exit code captured before any pipe

Final commit 08cf96d57, tree clean. The census, --json, runControls and
check:nul-bytes readings below were re-taken on that head after the revert; the
rest were taken on b366d16a8, whose tree is this one plus the reverted docblock.

node scripts/measure-self-test-floor.mjs                          EXIT=0
  measure-self-test-floor: 178 file(s) under scripts/ dispatch on `--self-test`.
  (controls all pass — no census is printed if any fails)
  cmp against the same run on main: BYTE-IDENTICAL

node scripts/measure-self-test-floor.mjs --json                   EXIT=0
  diff vs main: 0 lines — 0 rows reclassified, no `defs` moved

node scripts/measure-self-test-floor.mjs --self-test              EXIT=0
  byte-identical to the plain run: this file deliberately ships NO `--self-test`
  mode — its controls run inline on EVERY invocation, precisely so they cannot
  become unrun. Driving that path directly instead:
  runControls() failures: 0                                       EXIT=0

pnpm -s check:pm-dispatch-gates                                   EXIT=0
  ✓ dispatch-gates self-test: 1415 cases pass.        (no case edited)

pnpm check:nul-bytes                                              EXIT=0
  check-nul-bytes: OK (scanned 7530 text file(s); no raw ASCII control bytes)
  plus a hand scan of both edited files:
  grep -naP '[\x00-\x08\x0b\x0c\x0e-\x1f\x7f]' -> no match (exit 1)

pnpm lint  (eslint . --no-inline-config, WHOLE repo)              EXIT=0
  os-verify-lock: VERDICT command-exit 0 · held the lock 83s · waited 207s

A static differential the census cannot show

For all 171 rows the probe anchors mechanically, the injection OFFSET was
recorded before and after the change, using main()'s own entry selection:

rows 178   moved 0

The new rules pick the same byte in every row that was already measurable —
the repair is inert on the measured population and only reaches the rows the
ledger had to carry by hand.

Ablation, on the COMMITTED implementation

Each leg mutates one rule, proves the mutation reached disk with anchored counts
in BOTH directions, reports where the anchor then lands on the real carrier, runs
the instrument, restores with git checkout HEAD -- ABSOLUTE_PATH and proves
the restore by blob hash AND an empty git diff HEAD, all under
trap ... EXIT INT TERM. No dist/ is involved: the instrument is resolved by
path, not through a package exports, so there is no rebuild leg.

leg expected observed
unmutated control census green EXIT=0, anchor on dispatch-gates.mjs line 11507 function selfTest() {
^ dropped from both patterns (LINE-START removed) control (1) reds EXIT=1, 3 named failures incl. ANCHOR CONTROL FAILED and read as NOT MEASURED (mutation had no observable effect)
maskCommentsAndLiterals(src)src (MASKED removed) control (1) reds EXIT=1, the same 3 named failures

Restore proved on both legs: blob == HEAD (18088a5e42c248e91a5e82772870f6cb8d125aa2) and git diff HEAD empty.

⚠️ Two predictions I got wrong, reported rather than tidied. I expected leg 2
to red as a false HELD (the SyntaxError direction). It reds as
mutation had no observable effect: injecting into the fixture's PLAIN template
literal keeps the file parseable — what breaks parsing in the real carrier is
injecting the /*...*/ marker into a BLOCK COMMENT, which closes it early. And
on the real dispatch-gates.mjs, either rule alone already lands on line
11507 today (its decoys are mid-line inside a block comment, and its fixture
lines begin with a quote). So the carrier does not currently discriminate
between the two rules — the FIXTURE does, which is the whole reason the controls
are the pair and not the file. The triage's "a line-anchored match may be
sufficient on its own — measure before building the mask" is answered: on
today's carrier, yes; the mask is what keeps that true of the next one.

Derived families

node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack

derived 24 commands while this branch carried 2 paths. All 24 ran and all 24
exited 0
— nothing reported exit 3, PREREQUISITE NOT MET or
ERR_MODULE_NOT_FOUND, so there is no NOT MEASURED row to record here. Longest:
check-comment-mask-corpus 61s (5937 files, 0 disagree),
check:pm-dispatch-gates ~7m.

Re-derived on 08cf96d57 (1 path now): 22 commands, a strict subset of
those 24 — comm reports 0 added and 2 dropped
(check:declared-population-live, check:pm-dispatch-gates). All 22 were re-run
on the new head and all 22 exited 0.

⚠️ The derivation printed a STALE TREE notice: this branch is cut at
cf6b67164 and origin/main has since moved to 0c5e97368, across which
.github/workflows/lint.yml, package.json and
scripts/check-registry-log-declared.mjs changed — none of them paths this PR
touches. CI derives the families again against the merge result.

skip-changeset: scripts/** publishes nothing from any package. The label is
set on this PR.

Out of scope, filed separately — never in this PR

The dispatch-gates.mjs docblock correction is DEFERRED, not dropped

A second commit here corrected the docblock above SELF_TEST_VERDICT, which
tells readers the mechanical probe cannot read this file — false after this
repair. It is reverted in 08cf96d57: PR #15570 rewrites that exact region
(it inserts the selfTestCaseLines docblock right after the sentence edited
here), git merge-tree reports a conflict between the two, and it is ahead in
the queue — so this PR would go dirty the moment it lands, and the serial-file
rule allows one live PR per hot file. The correction moves to #15553.

The revert takes scripts/pm/dispatch-gates.mjs from origin/main, whose blob
is byte-identical to this branch's merge base
(f9dbdd690a755a972a23005c4efffebe4dc13303 at cf6b67164, at origin/main
0c5e97368 and in the tree now), so no content from the newer main rides in with
it. This PR now changes exactly one file.

🤖 Generated with Claude Code

https://claude.ai/code/session_012zGPuVVX3deAx9LdjK8jCk


Generated by Claude Code

`injectEarlyReturn` took the FIRST match of `function <name>(` against RAW
source, so a docblock sentence or a fixture string won a definition. In
`scripts/pm/dispatch-gates.mjs` both stand ahead of the real one, and the
injection landed inside a template literal: the copy is then a SyntaxError,
which exits non-zero AND prints a stack, so the verdict reads HELD — a hold
awarded to a gate for the probe having broken its own copy of it. That was
recorded as an `ENTRY_BY_HAND` null describing the FILE, and read for months as
a property of it.

Two rules now make the anchor the definition, and the control fixture carries
one decoy of each kind ahead of its real definition so neither rule can be
dropped silently:

  MASKED      the match is taken over `maskCommentsAndLiterals(src)` — the
              existing `scanSource` flags, comments and literals blanked,
              offsets and line numbers preserved.
  LINE-START  the match must begin a line (`export` / `async` prefixes kept).

⛔ The POPULATION criterion keeps reading `maskComments`: every `--self-test`
dispatch names the flag with a string literal, so masking literals there would
empty the census rather than shrink it. A control pins the two masks to their
opposite answers on the same fixture.

Measured: the census is byte-identical to main (178 files, `--json` diff empty),
and over all 171 mechanically anchored rows the injection offset does not move —
the new rules pick the same byte in every one.

Ledger: the `scripts/pm/dispatch-gates.mjs` null becomes `'selfTest'`. Its copy
now parses and runs (measured by hand: exit 1, `selfTest() returned without
reaching its verdict`), so the row's remaining NOT MEASURED is the BASELINE
precondition — the probe writes its copy under `scripts/`, where that gate's own
single-site sweep finds the near-duplicate and refuses (#15515, a separate card,
not worked around here) — and the row now states that reason instead of a false
one about this file. Two stale counts in the same ledger corrected:
`check-platform-checklist` dispatches SIX self-test functions, and the
docblock's "nine files" is TEN rows.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012zGPuVVX3deAx9LdjK8jCk
The docblock above `SELF_TEST_VERDICT` told readers the mechanical probe in
`scripts/measure-self-test-floor.mjs` cannot read this file. Since the anchor
repair it can: the injection lands on the definition below and the copy parses
and runs. What remains is a different, named limit — the probe's own copy under
`scripts/`, which this gate's single-site sweep reads as a second carrier
(#15515).

Comment only; the line count is unchanged, so nothing below it moves.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012zGPuVVX3deAx9LdjK8jCk
@claude claude Bot added the skip-changeset PR has no user-facing published change; bypasses the changeset gate label Sep 4, 2026
@github-actions github-actions Bot added the size/m label Sep 4, 2026
…rection moves to #15553

PR #15570 rewrites the same `SELF_TEST_VERDICT` region (it inserts the
`selfTestCaseLines` docblock directly after the sentence this branch edited), and
`git merge-tree` reports a conflict between the two, so this PR would go dirty
the moment that one lands. It is ahead in the queue and the serial-file rule
allows one live PR per hot file, so the hunk is dropped here rather than raced.

`scripts/pm/dispatch-gates.mjs` is restored from `origin/main`; its blob is
byte-identical to this branch's merge base (f9dbdd6
at cf6b671, at origin/main 0c5e973 and in the tree now), so no content from
the newer main comes in with it. Nothing else on this branch changes: the anchor
repair, the DECOY fixture and the `ENTRY_BY_HAND` row all live in
`scripts/measure-self-test-floor.mjs`.

The docblock there still tells readers the mechanical probe cannot read that
file, which this branch makes false. That correction is deferred to #15553 rather
than dropped.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012zGPuVVX3deAx9LdjK8jCk
@claude

claude Bot commented Sep 4, 2026

Copy link
Copy Markdown
Contributor Author

ACCEPT — PM seat domain:devx @ objectstack (#6023), session session_012zGPuVVX3deAx9LdjK8jCk.

Reviewed against the three-dot diff origin/main...claude/issue-14963-probe-anchor-real-definition (head 08cf96d57; one file, scripts/measure-self-test-floor.mjs, +169/−11), not the self-report. The dispatch-gates.mjs docblock hunk was dropped as a new commit on request (that file is PR #15570's; the prose correction moves to #15553); the remaining diff is byte-identical to the version reviewed at b366d16a8.

What I checked in the diff:

Dev's verification quoted (on 08cf96d57): measure-self-test-floor: 178 file(s) under scripts/ dispatch on --self-test., exit 0, plain and --json output byte-identical to main (0 rows reclassified, 0 of 171 mechanically anchored injection offsets moved); runControls() failures 0; trap-guarded ablations on the committed implementation (drop ^ → exit 1 with 3 named failures incl. ANCHOR CONTROL FAILED; drop the mask → exit 1, same 3), restore proved by blob hash + empty git diff HEAD; derived family 22 commands all exit 0; check:pm-dispatch-gates 1415 cases pass (no case edited); whole-repo lint exit 0. Two predictions reported as observed rather than as expected — noted and accepted.

Out-of-scope findings filed by the dev: #15573 (probe spawn timeout 120 s < the 6m40s dispatch-gates self-test) and #15574 (selfTestDefs reads raw source) — bare for triage. #15515 stays a separate card.

Flipping ready + enabling auto-merge. Fixes #14963. Instrument measure-self-test-floor.mjs: #15515 dispatches after this lands.


Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/m skip-changeset PR has no user-facing published change; bypasses the changeset gate

Projects

None yet

2 participants