Skip to content

test(plugin-security): compile the 89 test files no tsc program read - #13395

Merged
os-elon merged 3 commits into
mainfrom
claude/issue-13176-plugin-security-test-typecheck
Aug 30, 2026
Merged

test(plugin-security): compile the 89 test files no tsc program read#13395
os-elon merged 3 commits into
mainfrom
claude/issue-13176-plugin-security-test-typecheck

Conversation

@os-elon

@os-elon os-elon commented Aug 30, 2026

Copy link
Copy Markdown
Collaborator

Fixes #13176

@objectstack/plugin-security advertised "typecheck": "tsc --noEmit && tsc --noEmit -p tsconfig.scripts.json" while no tsc program in the package read a single one of its 89 test files — 1636 tests, in the package whose suites pin refusal behaviour. This adds the sibling tsconfig.test.json the AGENTS.md "Build & Test" section prescribes, names it in typecheck, and repairs the residue rather than ledgering it.

The hole, measured per program (not in aggregate)

A single combined zero cannot tell "excluded" from "the grep was wrong", so each program was measured separately with tsc --noEmit --listFiles, workspace closure built first, on origin/main at aa16721b6:

program files *.test.ts in it
tsconfig.json (build) 460 0exclude names **/*.test.ts
tsconfig.scripts.json 310 0include is scripts/**/*
tsconfig.test.json (new) 591 89 — the same grep, non-zero

The 89 is the positive control: the same pattern that answers 0 twice answers 89 once, and 89 is also what find reports on disk.

The card's "11 errors across 6 files" — re-measured, and it did not move

Measured at 1a540e82b in the card, re-measured here at aa16721b6: the same 11, in the same 6 files, at the same per-file counts. Under the vitest-matching module semantics this config adopts, 2 of the 11 are the CHECK rather than the code and disappear: TS1470 (import.meta in a file "which will build into CommonJS output" — a file vitest runs as ESM every day) and TS2550 (Array.prototype.at against a lib older than es2022, on Node >= 22). That is the same subtraction packages/spec and packages/rest made.

The remaining 9 are all repaired in this PR, so the layer compiles at ZERO and there is no ledger file at all:

file errors repair
explain-engine.test.ts 5 (TS2532 x2, TS18048 x3) ?. on contributors / rules, which are optional in the pre-parse z.input layer shape — an absent member then reaches the matcher as undefined and the expectation still fails loudly
rls-pushdown-limits.test.ts 2 (TS7006) the spy was ReturnType(typeof vi.spyOn), which instantiates that generic's own parameters and leaves mock.calls untyped; named as MockInstance(typeof console.warn) instead of annotating each callback
authz-matrix-gate.test.ts 1 (TS2345) RLS_DENY_FILTER is Record(string, unknown), so the sentinel arrived as unknown; converted once at the declaration
objects/default-permission-sets.test.ts 1 (TS2345) the drift pin compares its list against names read off the shipped schemas in BOTH directions, so the local is string[] rather than the literal union

⛔ Nothing was silenced: no strict loosened, no blanket any, no @ts-nocheck, no @ts-expect-error, no deleted assertion. pnpm --filter @objectstack/plugin-security test89 files, 1636 tests, all passing — after the repairs.

Why no test-typecheck-debt.json

At zero residue a bare tsc --noEmit -p tsconfig.test.json is the strictly stronger gate: any error is red immediately, with no ledger to be added to. Wiring scripts/check-test-typecheck.mts would cost this package a tsx dependency and two more scripts to hold an empty ledger. This is the call packages/metadata-core, packages/metadata-fs and packages/triggers/trigger-record-change already made in-tree, and each of their configs states it. The moment this package acquires residue that cannot be fixed in its own PR is the moment to wire the shared script — not before.

Reverse verification: the new program is not as blind as the old one

Predicted before running: the new leg goes red, the old one stays green on the same tree. A deliberate TS2322 was appended to src/audience-anchors.test.ts — a file that carried none of the 11:

BEFORE: hash=94d41db7b07e2ea98d05786b76015b3d324d9648 markers=0
AFTER:  hash=9c92f0e0233b397eff5b4af2bc0a95ad20b58354 markers=1
MUTATION CONFIRMED ON DISK (hash moved, injected text present 1 x)
NEW_PROGRAM(tsconfig.test.json) EXIT=2  errors=1
  src/audience-anchors.test.ts(168,14): error TS2322: Type 'string' is not assignable to type 'number'.
OLD_PROGRAM(tsconfig.json)      EXIT=0  errors=0
RESTORED: hash=94d41db7b07e2ea98d05786b76015b3d324d9648 markers=0
GIT_DIFF_HEAD_OUTPUT_BYTES=0
GIT_STATUS_PORCELAIN_LINES=0

Same bytes, same command, two answers — which is the whole point of the card. The restore is proven by state (blob hash back to the HEAD blob, marker count 0, empty git diff HEAD), not by an exit code.

A second, unplanned demonstration arrived for free: a comment edit in this PR put a */ inside a block comment, and typecheck reported 8 syntax errors in default-report-sink.test.ts — a file that, one commit earlier, no program would have read.

Two repo-level files, each demanded by its own gate

Declared on the issue before either was edited, with the measurements.

scripts/check-type-check-coverage.mjs — the TEST_DEBT entry is deleted (graduation). The gate names the remedy itself: "has a TEST_DEBT entry but has no test files -- it graduated; delete its entry". The entry stood at errors: 11, re-measured at exactly 11 before repair, so it was retired against a measurement. The ledger moves in the tightening direction only:

before  18 package(s) hide their own tests from tsc (1013 files hidden, 1186 frozen raw errors)
after   17 package(s) hide their own tests from tsc ( 924 files hidden, 1175 frozen raw errors)

-1 package, -89 hidden files (this package's, exactly), -11 frozen errors (this entry's, exactly). ⛔ Neither that script's DEBT nor its TEST_DEBT ledger is raised.

scripts/check-type-source-resolution.mjs — this package's registry entry is re-baselined, +5 deps. All five are annotated via tsconfig.test.json by the gate's own failure text — the program this change onboarded — which is condition 1 of the limb that gate's doc-block opens to a package ("a package whose typecheck script begins naming a tsconfig.test.json has moved the set BY DEFINITION"). paths is barred rather than merely unused: the same doc-block records PR #12570 measuring that route on packages/rest as taking its test layer from 37 to 42 errors, the +5 being other packages' source diagnostics billed to a ledger those packages cannot pay down. Condition 2, the numbers, stated in the registry beside the entry and here:

        before (origin/main @ aa16721b6)   after (this branch)
programs            94                            95
packages            77                            77
entries             54                            54
package-dep pairs  238                           243

+1 program, +0 entries, +5 pairs. --list before was measured in a dedicated comparison worktree at origin/main, not assumed.

Four in-tree notes this change falsifies, corrected

Each reasons from the hole being closed, and each is a receipt for what a hidden layer costs:

  • default-report-sink.test.ts and permission-set-projection.test.ts both said a @ts-expect-error here "evaluates never". It would now be live; the reason is corrected rather than the practice (the compile-time half stays with pnpm check:optional-error-sink).
  • explain-engine.test.ts explained a normalization by "keeping the test-layer TEST_DEBT flat" — that ledger entry is gone.
  • seed-write-refusal.test.ts records writing __dirname instead of import.meta.url to keep the TEST_DEBT ratchet from going 11 to 12 — a source file shaped around a program whose verdict nothing ever ran. The code is left alone (__dirname works under both); the note now says the reason is spent.

Verification

Gate family derived at edit time with node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack, and re-derived after the file set grew (the two scripts pulled in 7 more families plus the gate-script convention pair). All runs below are on 873284a46, the final commit, with exit codes captured before any pipe.

Green (exit 0): check:type-check-coverage · check:type-source-resolution · check:cross-package-test-inputs · check:doc-authoring · check:logger-receiver-detach · check:objectql-double-limit · check:override-consistency · check:page-declaration-shape · check:published-files · check:slot-lookup · check:test-source-alias · check:where-matcher · check:engine-double-contract · check:query-options-erasure · check:nul-bytes · check:i18n · check:i18n-stale-fill · check:agent-test-spelling · check:bash32-floor · check:cli-command-ids · check:entry-guard · check:parse-guard · check:pnpm-filter-targets · check:watch-hint-literal · check:pm-dispatch-gates · scripts/pm/bare-root-worklist.mjs --self-test · check-changeset-fixed · check-ci-filter-parity · check-comment-mask-adoption · check-keyed-text-bounds · check-osv-exemptions · check-plugin-teardown-shape · check-shard-attestation · check-undeclared-dep-imports · docs-audit/check-affected-docs · docs-audit/check-drift-comment · --filter @objectstack/spec check:llms-txt. Both edited scripts run their own --self-test first inside the pnpm check:* invocation, so both halves are covered by those two zeros.

pnpm lint (eslint . --no-inline-config, full repo, not narrowed): exit 0.
pnpm --filter @objectstack/plugin-security typecheck (all three programs): exit 0.
pnpm --filter @objectstack/plugin-security test: 89 files / 1636 tests, all pass.

NOT MEASURED (prerequisite not met — neither a red nor a finding):

  • check-test-completeness — exit 3, needs a saved turbo run test log; the script's own text prescribes recording it this way when the family is run locally.
  • check:dual-build-cjs-loads — exit 3, reads built output for 12 packages with no dist/; needs a full pnpm build.
  • check:type-check-debt — the RATCHET half, deliberately left to CI and declared here rather than skipped silently. It needs the whole workspace built plus a per-entry tsc re-measure, which does not fit one foreground run; the STRUCTURAL half that judges this PR's deletion is green above, and the deletion gives that ratchet strictly less to re-measure. check:i18n's prerequisite WAS cleared (10-package build) rather than declared, and it passes.

Out of scope, filed rather than folded

#13394 — two exported declarations in this package are wider than any value they hold (ExplainDecision.layers as the z.input shape; RLS_DENY_FILTER as Record(string, unknown)). Three of the nine errors repaired here are that class landing in the tests. Filed unassigned as an observation; narrowing a shipped declaration is not this card's business.

skip-changeset applies: no published behaviour changes. The only package.json edit is the typecheck script.


Generated by Claude Code

claude added 3 commits August 30, 2026 08:21
The package's `typecheck` script ran two programs, and neither read a
single test file: `tsconfig.json` names `**/*.test.ts` in `exclude`, and
`tsconfig.scripts.json` selects `scripts/**`. Measured per program with
`tsc --noEmit --listFiles`: 460 files / 0 tests, and 310 files / 0 tests.

Add the sibling `tsconfig.test.json` (the AGENTS.md "Build & Test"
pattern) and name it in `typecheck`. Module semantics match how vitest
executes the layer, strictness is inherited untouched. The 9 residual
errors are repaired rather than ledgered, so the layer compiles at zero
and no `test-typecheck-debt.json` is needed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012WkdHQwHr2KQmaX7P1BHzi
…s type-source registry

Both edits are demanded by the gates themselves once the package's test
layer enters a tsc program. check-type-check-coverage names the deletion
("it graduated; delete its entry"); check-type-source-resolution's five
new deps are all annotated `via tsconfig.test.json`, the program this
change onboarded, which is the re-baseline limb its doc-block opens to a
package. Numbers stated in place: 94 to 95 programs, 238 to 243 pairs.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012WkdHQwHr2KQmaX7P1BHzi
Each of them reasons from the hole being closed: two say a
`@ts-expect-error` here would evaluate never, one explains a normalization
by "keeping the test-layer TEST_DEBT flat", and one records writing
`__dirname` instead of `import.meta.url` to keep a ratchet from moving.
All four are now false or spent; the reasons are corrected rather than
deleted, because the cost of a hidden layer is the lesson.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012WkdHQwHr2KQmaX7P1BHzi
@os-elon os-elon added the skip-changeset PR has no user-facing published change; bypasses the changeset gate label Aug 30, 2026 — with Claude
@github-actions

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

Nothing in this diff resolved to a documentable surface (no symbol, route or SDK anchor derived from 0 changed package(s)), so this run has no opinion about the docs.

What this run could not see
  • a page that states a rule by its inputs shares no identifier with the emitter that implements the rule, so an emitter-only diff cannot list it — not on this run and not on any run. Measured on fix(driver-sql): emit varchar(maxLength) for a text field a declared index keys on #11430: content/docs/protocol/objectql/types.mdx documents the text-family column mapping by the ObjectQL type names it maps FROM (text / textarea / html) while the diff changed createColumn; it went unlisted, and it was the page that diff falsified, in four places. No shared token exists to detect this on, so a rule your change carries has to be re-read by hand in the pages that restate it.

Coarse fallback — 0 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json 83546d97053009cc84b90e1addeee7163d99911cpackageMentionDocs.

@github-actions github-actions Bot added dependencies Pull requests that update a dependency file tests labels Aug 30, 2026
@os-elon
os-elon marked this pull request as ready for review August 30, 2026 09:10
@os-elon
os-elon enabled auto-merge August 30, 2026 09:10
@os-elon
os-elon added this pull request to the merge queue Aug 30, 2026
Merged via the queue into main with commit a68c612 Aug 30, 2026
38 checks passed
@os-elon
os-elon deleted the claude/issue-13176-plugin-security-test-typecheck branch August 30, 2026 09:28
os-elon pushed a commit that referenced this pull request Aug 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dependencies Pull requests that update a dependency file size/m skip-changeset PR has no user-facing published change; bypasses the changeset gate tests

Projects

None yet

2 participants