Skip to content

test(runtime): migrate the notification-conformance fixture onto the shared read-refusal capture - #13352

Merged
os-trump merged 1 commit into
mainfrom
claude/issue-13325-migrate-inline-refusal-capture
Aug 30, 2026
Merged

test(runtime): migrate the notification-conformance fixture onto the shared read-refusal capture#13352
os-trump merged 1 commit into
mainfrom
claude/issue-13325-migrate-inline-refusal-capture

Conversation

@os-trump

Copy link
Copy Markdown
Collaborator

Fixes #13325

Nothing here was red and nothing became red. This is dead suppression plus stale prose.

What was dead — measured, not argued

packages/runtime/src/notification-schema-conformance.integration.test.ts carried its own copy of the expected-read-refusal capture (captureExpectedAbsentTableNoise) and wrapped the engine's error channel only.

Since #13273 landed as PR #13327, ObjectQL.reportFindFailure picks the level from the cause: a read whose table was never provisioned is logged at debug, with a reason: 'table-not-provisioned' meta and no stack, while every other read failure keeps error with the stack. That is exactly — and only — the class this capture is declared over, so its recognition arm could no longer match a single frame.

Measured on this tree at the merge base (0783d7b80), by instrumenting the inline capture with counters, running the suite, then restoring the file:

counter value
driver refusals recognised and withheld 63
engine error channel invocations, any message 0
engine error frames matching the arm's predicate 0
engine debug channel invocations 417
engine debug frames whose msg is Find operation failed 63
those same frames satisfying the arm's OWN predicate 63

The last row is the positive control for the zeros above it: the zero is not a broken predicate. The identical recognition rule matches 63 of 63 frames — it was simply reading the channel the engine stopped using. The file stayed green throughout because everything it asserts is fed by the DRIVER channel, which #13273 never touched. So its engine-side suppression was dead code that read as live protection.

Instrumentation hygiene: the mutation was confirmed on disk before the run (injected marker present, removed anchor at zero occurrences, blob hash differing from the HEAD blob), and the restore was proven the same way afterwards — blob hash back to 2c4e4aecd, zero marker residue, git diff HEAD empty, git status --porcelain empty.

The migration

The file now uses the shared captureExpectedReadRefusals (#10629), which already proxies both error and debug:

  • captureExpectedAbsentTableNoise() — 70 lines of per-fixture copy — is deleted;
  • noise.install(driver) becomes noise.captureDriver(driver);
  • noise.wrapEngine(...) becomes noise.captureEngine(...);
  • the afterAll assertion moves from the per-table noise.withheld.has(table) loop to expect(noise.silentChannels(ALWAYS_READ_AUTHZ_TABLES)).toEqual([]).

ABSENT_AUTHZ_TABLES and ALWAYS_READ_AUTHZ_TABLES are unchanged, and packages/runtime/src/expected-read-refusal-noise.ts is not edited by this PR — this card is a consumer of it.

The assertion is strictly stronger, never weaker. The loop it replaces read the DRIVER channel only, which is precisely why it kept passing while the engine half was dead. silentChannels() requires BOTH channels to have fired for each of the five always-read tables, and a silent one names itself in the diff.

Alive after — reverse verification

Direction predicted before running: RED, naming the engine channel for the five always-read tables.

Ablating the shared helper's debug arm — restoring its pre-#13327 shape — and re-running the migrated file:

FAIL  src/notification-schema-conformance.integration.test.ts
AssertionError: expected [ …(5) ] to deeply equal []
+   "the engine's 'Find operation failed' frame for 'sys_user' was never emitted",
+   "the engine's 'Find operation failed' frame for 'sys_member' was never emitted",
+   "the engine's 'Find operation failed' frame for 'sys_user_position' was never emitted",
+   "the engine's 'Find operation failed' frame for 'sys_user_permission_set' was never emitted",
+   "the engine's 'Find operation failed' frame for 'sys_position' was never emitted",

 Test Files  1 failed (1)
      Tests  8 passed (8)

The eight test cases still pass under the ablation; only the newly live engine assertion in afterAll goes red. That is the exact shape claimed above — the driver-fed half is untouched, the engine-fed half is now load-bearing.

Ablation hygiene, both legs: mutation confirmed on disk (marker injected once, the debug-arm anchor at zero occurrences, blob 982555d52 against the HEAD blob 064a19632), restore confirmed the same way (blob back to 064a19632, zero residue, git diff HEAD empty, clean git status --porcelain). No rebuild was needed for this leg: the helper is a relative same-package import vitest resolves to source, not a package exports entry pointing at dist. The engine under test IS resolved through dist, so @objectstack/objectql was built first and its artifact was confirmed to carry table-not-provisioned before any measurement was taken.

Re-measured header counts

The header's previous numbers were a statement about origin/main at #10380 time. They are re-derived here from a run on this tree, and the prose no longer claims the engine frame arrives on error. Command:

pnpm --filter @objectstack/runtime exec vitest run \
  src/notification-schema-conformance.integration.test.ts

Read off the migrated capture itself (totalRefusals() / totalEngineFrames() / refusals / engineFrames):

table driver refusals withheld engine debug frames withheld
sys_user 12 12
sys_member 12 12
sys_user_position 12 12
sys_user_permission_set 12 12
sys_position 12 12
sys_setting 3 3
total 63 63

silentChannels(ALWAYS_READ_AUTHZ_TABLES) returns []. Before the migration the engine tally was 0 for every one of those tables. The header states the two totals as PROSE rather than as a pin, and says how to re-derive them, so nobody copies them forward again.

Every existing assertion still passes

All measurements below were taken on the tree at 5293ddc76, this branch's final commit, with a clean working tree.

  • Migrated file: Test Files 1 passed (1), Tests 8 passed (8), exit 0. The six tables recorded on the driver channel are unchanged from the pre-migration run.
  • Migrated file plus the sibling whose boot it mirrors (notifications.hono.integration.test.ts) plus the shared helper's own two pin suites (expected-read-refusal-noise.channel-asymmetry.test.ts, cross-field-refusal-operand-withhold.test.ts): Test Files 4 passed (4), Tests 28 passed (28), exit 0.
  • Whole affected package, pnpm --filter @objectstack/runtime exec vitest run: Test Files 201 passed (201), Tests 2968 passed (2968), exit 0.
  • Repo-wide pnpm lint (eslint . --no-inline-config): exit 0, no narrowing.

Gate families derived from the real change set rather than recalled — node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack, which reports the change set as one path against merge base 0783d7b80 — and then run. Every one green, quoting each gate's own verdict line:

  • check:nul-bytesOK (scanned 7390 text file(s) … no raw ASCII control bytes)
  • check:cross-package-test-inputsOK: 24 package(s) read outside themselves, all declared
  • check:engine-double-contractOK — 709 pinned, 134 in the DEBT ledger, 3 exempt
  • check:where-matcher316 matcher(s) discovered … 0 silently-wrong and 0 unjudged … none new
  • check:query-options-erasureratchet holds: 67 unswept non-test site(s) … none new
  • check:test-source-aliasOK — 72 packages with tests scanned
  • check:type-check-coverageOK — 66/78 workspace packages type-checked
  • check:type-check-debt--re-measure: OK — 30 ledger entr(ies) re-measured in 234.3s, 1558 raw tsc error(s) total, none above its recorded number (surplus: none)
  • check:dual-build-cjs-loads102 published require entry point(s) across 66 package(s) load; 610 emitted CommonJS file(s) parse (run after a full turbo run build, 71 successful, 71 total)
  • check:logger-receiver-detach, check:objectql-double-limit, check:page-declaration-shape, check:published-files, check:slot-lookup, check:type-source-resolution — exit 0
  • scripts/check-ci-filter-parity.mjs, check-comment-mask-adoption.mjs, check-cross-package-test-inputs.mjs, check-keyed-text-bounds.mjs, check-plugin-teardown-shape.mjs, check-shard-attestation.mjs, check-undeclared-dep-imports.mjs, docs-audit/check-affected-docs.mjs, docs-audit/check-drift-comment.mjs — exit 0

Two readings are recorded as NOT MEASURED rather than folded into the green list:

  • node scripts/check-test-completeness.mjs — exit 3, with the gate's own refusal text: PREREQUISITE NOT MET — this gate grades a saved 'turbo run test' log, and no log was named, and running the family locally, record this gate as NOT MEASURED. Not a red, and nothing to repair.
  • pnpm --filter @objectstack/runtime typecheck exits 0, but that green says nothing about the edited file: packages/runtime/tsconfig.json excludes **/*.test.ts, and tsc --noEmit --listFiles returns 0 occurrences of the edited path. The hidden test layer of this package IS measured, by check:type-check-debt --re-measure, which re-ran the @objectstack/runtime TEST_DEBT entry and reported it not above its recorded number.

Scope

packages/runtime/src/metadata-list-ambient-vs-bare-transaction.integration.test.ts is deliberately untouched and is not in this diff. Its declared fault is Timeout acquiring a connection, which is not a missing table, so its frame legitimately stays on error and its withheld.engineFind assertion still fires today. It also cannot adopt the shared helper even in principle: that helper's predicate is keyed on a declared table name plus that table's no such table reason, and this fixture has neither. Repairing it by symmetry would break a working pin.

Changeset

None, deliberately. The diff is a single test fixture — no package publishes anything from it, nothing user-visible changes, and the file is excluded from this package's tsup entry and from its tsc program. The skip-changeset label carries that declaration.

Generated by Claude Code


Generated by Claude Code

…shared read-refusal capture

The file carried its own copy of the expected-read-refusal capture and wrapped
the engine's `error` channel only. Since #13273/#13327 `ObjectQL.reportFindFailure`
picks the level from the cause, so a read whose table was never provisioned --
which is every read this capture is declared over -- is logged at `debug`. The
inline recognition arm could therefore no longer match a single frame: measured
on this tree before the change, the engine `error` channel was invoked 0 times
while 63 `Find operation failed` frames arrived on `debug`, all 63 satisfying
that arm's own predicate. The file stayed green because everything it asserts is
fed by the driver channel, so the engine-side suppression was dead code reading
as live protection.

It now uses `captureExpectedReadRefusals` (#10629), which wraps both channels.
The `afterAll` assertion moves from the driver-only `withheld.has(table)` loop to
`silentChannels(ALWAYS_READ_AUTHZ_TABLES)`, which is strictly stronger: it
requires both channels to have fired for every always-read table. The header's
counts are re-measured on this tree rather than carried forward, and its prose no
longer claims the engine frame arrives on `error`.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TvqBFLRzXdSPcbusDoED9k
@github-actions

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

Nothing in this diff resolved to a documentable surface (no symbol, route or SDK anchor derived from 0 changed package(s)), so this run has no opinion about the docs.

What this run could not see
  • a page that states a rule by its inputs shares no identifier with the emitter that implements the rule, so an emitter-only diff cannot list it — not on this run and not on any run. Measured on fix(driver-sql): emit varchar(maxLength) for a text field a declared index keys on #11430: content/docs/protocol/objectql/types.mdx documents the text-family column mapping by the ObjectQL type names it maps FROM (text / textarea / html) while the diff changed createColumn; it went unlisted, and it was the page that diff falsified, in four places. No shared token exists to detect this on, so a rule your change carries has to be re-read by hand in the pages that restate it.

Coarse fallback — 0 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json 881f8d8e88a60ac73a9c3a376c48e679b849ab6cpackageMentionDocs.

@github-actions github-actions Bot added the tests label Aug 30, 2026
@os-trump os-trump added skip-changeset PR has no user-facing published change; bypasses the changeset gate size/m tests and removed size/m tests labels Aug 30, 2026 — with Claude

Copy link
Copy Markdown
Collaborator Author

✅ PM review — ACCEPT once CI is green; ⛔ not while it is running (#13325)

Undrafting now; arm follows on a complete green read. ⛔ No rework owed. Clause ② does not attach — one test fixture, nothing under packages/spec/src/**.

✅ Fences held, verified from the diff

  • metadata-list-ambient-vs-bare-transaction.integration.test.ts is not in the diff. Change set is exactly one file (changed_files: 1). That file's fault is a connection timeout, not a missing table, so its frame legitimately stays on error — leaving it alone was the whole point of naming it.
  • expected-read-refusal-noise.ts is not edited either. It was ablated and restored byte-identically as a measurement, which is the correct way to touch a file you must not change.

⭐ The dead-arm proof is the best measurement in this round

Instrumented at the merge base, per channel:

driverRefusals 63 · errorCalls 0 · errorFindFrames 0 · errorMatches 0
debugCalls 417 · debugFindFrames 63 · debugWouldMatch 63

The positive control for those zeros is the sharp part: the dead arm's own predicate matched 63 of 63 frames — on debug. So the zero is proven to be the wrong channel, not a broken predicate. Without that control, "0 matches" reads equally well as "the recognition logic is broken", and the repair would have been aimed at the wrong thing.

Alive-after: totalRefusals 63, totalEngineFrames 63, per table 12/12/12/12/12/3 on both channels, silentChannels() [].

⭐ The migration made the assertion STRONGER, not merely equivalent

afterAll moved from a per-table withheld.has(table) loop — driver channel only, which is precisely why it kept passing while the engine half was dead — to expect(noise.silentChannels(ALWAYS_READ_AUTHZ_TABLES)).toEqual([]), which requires both channels to have fired for each table and makes a silent one name itself in the diff.

That is the right outcome for a card about dead suppression: the replacement cannot go dead the same way, and it says so at the assertion site.

⭐ And the rotted number was fixed at the root, not re-transcribed

The header's counts are re-measured on this tree with the command that produced them, and then explicitly demoted:

⛔ Those two totals are PROSE, not a pin — they move with the routes this file drives, and re-deriving them means reading noise.totalRefusals() / noise.totalEngineFrames() off a run, never copying the numbers forward.

A count that rotted once will rot again; labelling it as prose and naming the accessor that re-derives it is a better fix than a fresher number.

Reverse verification ablated the shared helper's debug arm back to its pre-#13327 shape, predicted red, and got exactly the predicted red — five items naming the engine frames for sys_user, sys_member, sys_user_position, sys_user_permission_set, sys_position, with only the newly-live assertion failing and every driver-fed case staying green. ⭐ And the rebuild condition was reasoned rather than assumed: the helper resolves to source (same-package relative import), but the engine under test resolves through dist, so @objectstack/objectql was built first and its dist confirmed to carry the table-not-provisioned marker before any measurement. Missing that would have silently invalidated the whole exercise.

⚠️ Process incident — self-caught, repaired, and worth recording

Its first PR-page label extractor was written for the issue-page link form with no positive control on a PR page, so it read this PR's labels as empty when they were size/m and tests — and the whole-set label write then stripped both. A comparative read-back caught it and a union write re-hung all three.

✅ Verified from observed state, not the report: labels now read size/m · tests · skip-changeset — none missing, none extra.

⚠️ This is #13141's failure mode exactly — a blind label read feeding a read-modify-write that strips concurrent labels — which this lane already has a card for. ⭐ Crediting the handling: it built a comparative read-back, caught its own damage, repaired it, and reported it unprompted rather than leaving a silent strip behind. That is the behaviour that makes a self-inflicted incident cheap. The instrument lesson is the same one this round keeps producing: an extractor needs a positive control on the page shape it will actually run against, not on the one it was written for.

Also recorded: pnpm --filter @objectstack/runtime typecheck exits 0 but is NOT MEASURED for the edited file (tsconfig excludes **/*.test.ts; --listFiles returns 0 occurrences). The hidden layer is covered instead by check:type-check-debt --re-measure, which re-ran the @objectstack/runtime TEST_DEBT entry (217) and reported it not above its recorded number. check-test-completeness recorded NOT MEASURED with its own refusal text.


Generated by Claude Code

@os-trump
os-trump marked this pull request as ready for review August 30, 2026 06:11
@os-trump
os-trump added this pull request to the merge queue Aug 30, 2026
Merged via the queue into main with commit 2e0b7b1 Aug 30, 2026
48 of 49 checks passed
@os-trump
os-trump deleted the claude/issue-13325-migrate-inline-refusal-capture branch August 30, 2026 07:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/m skip-changeset PR has no user-facing published change; bypasses the changeset gate tests

Projects

None yet

2 participants