fix(tooling): a template-literal mode for the vocabulary gate's four shared textual scanners

[claude]

Fixes #14626

A nested template literal desynchronised the vocabulary gate's shared textual
scanners, and every code: position after it in that file went unplaced. This
adds a template-literal mode to all four primitives through ONE shared skipper,
makes the two undefined answers distinguishable, and re-derives both pinned
censuses on this tree.

The primitive population, re-derived by grep (not from the card's list)

On the branch base 4d0d9445a, grep -n "c === \"'\" || c === '\"' || c === ' + "" + '"finds the quote-skip at exactly FOUR sites, and the seek idiomwhile (i < src.length && src[i] !== quote) i += ... ? 2 : 1` at the four lines
beside them — the four the triage named, at this base's line numbers:

function	if (c === quote)	seek
scanTopLevel	1140	1143
enclosingOpeners	1548	1551
sliceBalanced	1566	1569
splitTopLevel	1584	1587

After the change, grep -c "!== quote) i +=" over the file returns 1, and
that one is legacySeekQuote inside --self-test — the positive control.

The helper's state machine, in five lines

skipStringLiteral(src, at) returns { end, closed }, where end is the index
of the CLOSING quote (the same post-condition the four seek loops had, so each
caller still advances past it with its own i += 1).

1. A STACK of frames, not a single mode: quote and template are string
   modes, interp is a code mode.
2. ' / " — walk to the matching quote; \ escapes the next byte.
3. ` — TEMPLATE: \ escapes, a bare ` closes, ${ pushes an
   interp frame with brace depth 1.
4. ${ … } — CODE: {/} move the brace depth (so `${ {a: 1}.a }`
   closes where it should), and a quote there pushes a NEW literal frame, which
   recurses through the same stack to any depth.
5. Running off the end with a frame still open returns closed: false — the
   only self-evident signature of a scan that lost its place.

The two undefineds, told apart — and CERTIFIED

enclosingOpeners now answers EVERY requested index with a tagged value:
{kind:'bracket', ch, at} · {kind:'top-level'} · {kind:'inside-string'} ·
{kind:'desync'}. The other three primitives take an optional scanReport()
(desynchronised, unterminatedAt, underflow, unbalanced), because their
return values have no room for a per-position tag.

inside-string is asserted only for a walk that ends health-certified: no
unterminated literal, no bracket underflow, an empty stack at EOF. Comments are
masked and brackets inside real literals are skipped, so a correct walk of
well-formed source ends empty; not ending empty is proof it did not. In a walk
that is not certified, every unvisited index is tagged desync — "cannot place
this, and cannot certify why" — rather than being called inside-string, which
would be the same over-claim one layer up.

--self-test: a fixture AND a positive control per primitive

The pre-fix legacySeekQuote is kept in test scope, with all four primitives
re-implemented around it differing in exactly the skip. Fixture:
`Unknown key${n > 1 ? 's' : ''} ${keys.map((k) => `\`${k}\).join(', ')} …` ``

primitive	fixture asserts	control (pre-fix algorithm) asserts
scanTopLevel	2 top-level ; past the template	legacy sees 0
enclosingOpeners	the code: after it is {-enclosed	legacy answers undefined
sliceBalanced	the whole argument list comes back	legacy truncates at the inner backtick
splitTopLevel	2 parts	legacy answers 1

Plus: inside-string on a code: written inside a string; desync on a
position behind an unterminated template; the skipper's two interpolation cases
(braces and quotes inside ${ … }); and the scanReport on all three.

Mutation on the committed tree. Deleting the ${-tracking branch from
skipStringLiteral (the skipper falls back to seek-to-matching-backtick)
reddens exactly the four primitive cases:

check-dispatcher-error-vocabulary --self-test FAILED:
  - scanTopLevel saw 0 top-level `;` past a nested template, not 2 …
  - enclosingOpeners answered {"kind":"desync"} for a `code:` after a nested template …
  - sliceBalanced returned "`a${xs.map((k) => `\`${k}\`` " for an argument list …
  - splitTopLevel cut a nested-template argument list into 1 parts, not 2 …

Mutation confirmed on disk before the run (grep -c on the deleted text: 1 → 0;
on the injected marker: 0 → 1; blob hash 9d4b8492 → 1dce95d3). Restored by
git checkout HEAD -- ABSOLUTE_PATH under an EXIT/INT/TERM trap and proven back:
blob hash 9d4b8492 again, git diff HEAD --name-only empty.

Censuses re-derived — tree drift separated from what the fix moved

Both readings come from the same sweep, run with the PRE-fix primitives and the
POST-fix ones over today's packages/** non-test source.

INLINE_LITERAL_EXPRESSION_CENSUS

field	pinned (#13790)	pre-fix, today	after	what moved it
filesScanned	2186	2187	2187	drift
objectLiteralCodeTokens	902	906	906	drift
anchorHits	346	349	349	drift
notObjectLiteral	74	74	70	fix
↳ parameterList	41	41	41	—
↳ insideAString	33	33	18	fix (split)
↳ behindScannerDesync	—	—	11	fix (new class)
ownedBySibling.objlitconst	40	40	41	fix
candidatesReachingReduction	216	219	222	drift + fix
declinedByReduction	214	217	218	drift + fix
↳ declinedTypeAnnotation	150	155	155	drift
↳ declinedRuntimeValue	64	62	63	drift + fix
livePositions	2	2	4	fix
distinctValues	5	5	9	fix
newVerdictRows	2	2	2	—
unregisteredWireCodesHiding	0	0	0	—
positionsBehindScannerDesync	2	2	11	re-attributed (below)
positionsRecoveredByTemplateMode	—	—	2	new

Newly PLACED positions, classified

Over the whole 906-token code: population, 19 positions go from unplaced
to placed. Two of them reach objlitexpr — exactly the two the card predicted:

position	classification
domains/automation.ts:1384 code: targetName === undefined ? 'required' : 'invalid_type'	ADR-0112 D6 field-addressed validation catalog. Same file, same shape, same two values as the :1125 instance the gate already saw ⇒ derives the site keys that already exist ⇒ 0 new verdict rows.
domains/automation.ts:1399 code: targetLabel === undefined ? 'required' : 'invalid_type'	Same class, same verdict, 0 new verdict rows.

The other 17 are objlit/objlitconst tokens (those shapes are not guarded by
enclosingOpeners, so they were never blind there) or type annotations. Two
enter objlithelper's candidate population, which is why localTwinCensus
moves by 2.

No new value surfaced anywhere. deriveSites diffed key-by-key across the
change: sites 59 → 59, unresolved 5 → 5, added [], removed [], per-shape
counts identical. So the ⛔ escalation clause (an unregistered value at a newly
placed position ⇒ report blocked) was not reached, and
packages/runtime/src/dispatcher-error-vocabulary.ts is untouched.

OBJECT_LITERAL_CODE_HELPER_BLINDNESS

measured.* and declarationFormClosure.* are historical deltas of #13233 /
#13226, not censuses of today's tree; re-checked rather than assumed —
objlithelper derives the same 29 sites and the same 5 unresolved before and
after — and left as they were, with that check written into the block.

localTwinCensus IS a census of this tree, and it had already drifted:

field	pinned (#13478)	pre-fix, today	after
candidatePositions	125	134	136
alreadyCoveredByObjlitconst	36	40	41
newlyReached	89	94	95
newlyReachedDistinctFileIdent	69	72	73
newlyReachedDistinctFiles	45	47	47
reduceToEmptyByClass.typeKeywordPosition	49	53	53
reduceToEmptyByClass.namedTypePosition	16	16	16
reduceToEmptyByClass.runtimeValueLocal	15	16	17
reduceToEmptyByClass.bindingWithoutDeclarator	9	9	9
reduce	0	0	0
newVerdictRows / unregisteredWireCodesHiding	0 / 0	0 / 0	0 / 0

The headline zero survives at both ends — the reducible subclass is still
empty — so no verdict row is owed. The instrument was the #13478 replica: the
real objlithelper branch with the resolveConstant fallback switched on,
driven through the real deriveSites; its zero is backed by the positive
control already in --self-test (resolveConstant still reduces
const c = cond ? A : B to both branches).

SCANNER_LITERAL_BLIND_SPOTS — the class this card did NOT close

The sweep found a second desync class and it is declared with numbers rather
than left to be rediscovered: a regex literal whose character class carries a
quote or a backtick. The live one is
packages/rest/src/error-response.ts's RELATION_DOES_NOT_EXIST, whose class
contains ", ' and a backtick; a textual walk opens a string the source does
not have.

files swept	2187
whole-file walks health-certified, pre-fix → post-fix	2108 → 2119 (+11)
files still desynchronised	68
code: positions unplaced pre-fix	86
↳ newly placed by the template mode	19
↳ certified genuinely inside a string	21
↳ still behind a desync	46
unregistered wire codes surfaced	0

⛔ Deliberately not fixed here: telling /re/ from division needs the preceding
token's grammatical class, which is lexer state this scan does not carry — a
heuristic would trade a blind spot that is now COUNTED for one that is not.
Filed as its own card. --self-test pins the declaration in the direction it
can fail: a regex-literal fixture must still come back desync, with the same
source minus the regex as the positive control, so the block cannot quietly
outlive the blind spot it declares.

Scope

• git diff --stat: 1 file changed, scripts/check-dispatcher-error-vocabulary.mjs.
• ⛔ scripts/check-error-code-casing.mjs — untouched. Confirmed by the diff
  and by pnpm check:error-code-casing in the gate family below.
• ⛔ packages/runtime/src/dispatcher-error-vocabulary.ts — untouched. No
  ledger row added or moved; no site or unresolved entry changed.
• ⛔ No shape grammar changed: objlit, objlitconst, objlittemplate,
  objlithelper and objlitexpr regexes are byte-identical.

Verification

The union below ran at HEAD 160c99d5 (git rev-parse --short HEAD), the
final commit on this branch — a merge of origin/main after the fix commit
0dc603cd2. The whole union was run again on that merged head rather than
carried over from the pre-merge run. Each exit code was captured by redirect
BEFORE any pipe.

node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack --commands scripts/check-dispatcher-error-vocabulary.mjs derives 18 commands from this
tree (objectstack-ai/objectstack at 160c99d51, --repo asserted and held —
the derivation is re-run from the merged head, never a git diff list fed in by
hand). All 18 run, all green except one NOT MEASURED:

node scripts/check-ci-filter-parity.mjs            exit=0
node scripts/check-cross-package-test-inputs.mjs   exit=0
node scripts/check-self-test-wired.mjs             exit=0
node scripts/check-shard-attestation.mjs           exit=0
node scripts/check-test-completeness.mjs           exit=3   NOT MEASURED
node scripts/check-whole-set-label-write.mjs       exit=0
node scripts/pm/bare-root-worklist.mjs --self-test exit=0
pnpm check:agent-test-spelling                     exit=0
pnpm check:bash32-floor                            exit=0
pnpm check:cli-command-ids                         exit=0
pnpm check:cross-package-test-inputs               exit=0
pnpm check:dispatcher-error-vocabulary             exit=0
pnpm check:entry-guard                             exit=0
pnpm check:parse-guard                             exit=0
pnpm check:pm-dispatch-gates                       exit=0
pnpm check:pnpm-filter-targets                     exit=0
pnpm check:ratchet-remedy-authority                exit=0
pnpm check:watch-hint-literal                      exit=0

node scripts/pm/dispatch-gates.mjs --ran RECORD_FILE on the --commands output
reconciles the run: "Run reconciliation — 18 derived, 18 run, 0 NOT-MEASURED,
0 UNRUN", exit 0.

check-test-completeness exit 3 is its own declared NOT-MEASURED branch, in
its own words: "Arrived here from the gate family scripts/pm/dispatch-gates.mjs
derives? … There is no local log to hand it, so the local reading for this gate
is NOT MEASURED. ⛔ It is not a red." CI tees a real turbo run test log into
it, so CI measures it.

Named by the dispatch and outside the derived family, run anyway:

pnpm check:error-code-casing   exit=0    (the ⛔ untouched gate — still green)
pnpm check:nul-bytes           exit=0    "OK (scanned 8042 text file(s) … no raw ASCII control bytes)"

The gate's own verdict lines:

check-dispatcher-error-vocabulary --self-test: 10 shapes + 282 assertions OK
  (vocabulary + #9098 door typing)                       [was 277 before this card]
check-dispatcher-error-vocabulary: OK — 59 unregistered code-stamping site(s),
  all classified; 1 awaiting a ledger entry (#8846).

Lint, narrowed and the narrowing measured (not "not run"):

• Population comes from eslint's own config, not from a guess about which files
  count; the diff is 1 file, so npx eslint --no-inline-config --format json scripts/check-dispatcher-error-vocabulary.mjs is the whole of it.
• File count read from that JSON: 1 file linted, 0 errors, 0 warnings,
  exit 0.
• Invariance: eslint.config.* states it runs with no parserOptions.project
  and no typed @typescript-eslint rules, so type-aware linting is off and a
  one-file diff cannot move the verdict on any untouched file. Repo-wide
  pnpm lint is CI's run.

🤖 Generated with Claude Code

https://claude.ai/code/session_01WLJQhde67SeTccsmnBVarV

---

Generated by Claude Code

[github-actions]

⛔ merge queue 构建失败 — 先分诊，再决定要不要重排

队列构建 33696604625 红了。队列跑的是全量套件（PR 侧 CI 只跑 affected 子集），
所以失败的测试可能在本 PR 没碰过的包里 —— 那不是重排能修的。每次盲目重排都会让排在后面的所有 PR 重建一轮。

失败的 job（日志抽取，best effort）：

• Test Core (1/6) — 失败步骤: Run this shard's tests

  @objectstack/cli:test:  FAIL   integration  test/run-dev-unbuilt-workspace.e2e.test.ts > the mirror direction: a reader that is never coming back > gives up and exits instead of waiting forever
    ↳ 失败原因: @objectstack/cli:test: AssertionError: the harness SIGKILLed the child — it was still alive at the ceiling. cap 180000 ms (RUN_TIMEOUT_MS, constant and load-independent by design); this child ran 1801
  

↳ 失败原因 是判读的关键：超时（Test timed out in … / Hook timed out in …）多半是负载/时序，不是本 PR 的回归；
断言（AssertionError: …）才指向真实的行为改变。两者的 FAIL 行长得一模一样，只有这一行能区分。

⚠️ 断言这一侧有一类例外，判据是断言在测什么，不是它是不是 AssertionError。 断言的对象是产品行为（一个值、一个形状、一次拒收）⇒ 照上面读：真实的行为改变，去查，⛔ 不要重排掉；
断言的对象是这次实验自身的有效性前提（跑完的耗时、负载下的先后、任何只在时间预算内才成立的条件）⇒ 它跟超时是同一类，同样对负载敏感，重排一次是合法的判别手段。
识别是机械的：断言的消息或它比较的值本身点名了一段时长、一个时间戳、一个耗时计数。实测过的一对 —— AssertionError: SecurityPlugin.init() ran: expected false to be true 测的是产品行为（真回归）；
AssertionError: this run took over a second, so second-precision stamps could have differed too: expected 1006 to be less than 1000 测的是实验前提：它守护的那条不变式当时是绿的，同一个 head 原样重排一次即成功。
穿着 AssertionError 外衣的时间测量，仍然是时间测量。（⛔ 这只改「怎么读一次红」，不改「哪些测试可以重排」——后者由别处管。）

跨 PR 相同签名（24h，按失败测试文件聚合）：

• test/run-dev-unbuilt-workspace.e2e.test.ts — 24h 内已弹出 15 个 PR（fix(objectql,runtime,rest): store a serializable manifest projection in the package registry #14499、fix(plugin-sharing): let system writes materialize sharing rules — drop the isSystem skips in bindRuleHooks (#13533) #14528、fix(devx): guard the last two run-on-import gate scripts — KNOWN_IMPORT_UNSAFE 2 to 0 #14593、test(driver-sql): give the seven unconditionally-live hooks an explicit 60_000 budget #14629、fix(cli): let the dev artifact door own stack-declared security metadata on a host config #14631、fix(metadata): drop the leftover boot-time debug probe from MetadataPlugin.init #14649、fix(cli): walk bulkActions, validation messages and datasets in i18n extract, so the coverage ratchet can see them #14651、fix(objectql,metadata): derive a view container's object through the shared helper, so the row's own name is LAST at every SOURCE registrar (#14399) #14665、fix(cli): remove the ghost field types from generate.ts's three vocabularies #14675、test(runtime): measure which declaration copy the security/sharing seeders consume on an artifact boot with an engine #14687、Declare the scripts/ population two Silent gate families walk but never named #14692、test(cli): make the unread-reader ceiling a load-independent constant at RUN_TIMEOUT_MS #14715、fix(devx): partition os-regen-merge.sh's step-1 conflict message by class #14733、fix(spec): send a top-level flow trigger to the START node config, not to a type rename #14735、fix(tooling): a template-literal mode for the vocabulary gate's four shared textual scanners #14739），分属 11 条投机栈 ⇒ 11 次独立命中。汇总 issue：Queue-flake anchor: test/run-dev-unbuilt-workspace.e2e.test.ts #14752（已刷新）
• ⚠️ 24h 评论账本没读完（超过 5 页仍未读到窗口尽头），所以上面的「不同 PR 数」是下界，不是全量。

历史信号：

• 本 PR 过去 24h 无队列失败记录（首次）。
• 过去 24h 队列共有 30 个失败构建（不含本次）。

分诊清单：

1. 失败测试在本 PR 改动的包里 → 真回归，修 PR。
2. 失败测试与本 PR 无关 → 看上面的「跨 PR 相同签名」；已有汇总 issue ⇒ flaky/环境问题实锤，去那张 issue 上谈，修好前重排只会再烧一轮全队列。
3. 两者都不是 → 可能与同组 PR 语义冲突；等前面的 PR 落地或失败出队后再重排一次即可，不要连续重排。

---

Generated by Claude Code · merge-queue-triage workflow (#4859)