Skip to content

feat(spec): refuse a null comparand in the ordering positions — $gt / $gte / $lt / $lte (#14080) - #14425

Merged
os-musk merged 3 commits into
mainfrom
claude/issue-14080-ordering-null-comparand-refusal
Sep 2, 2026
Merged

feat(spec): refuse a null comparand in the ordering positions — $gt / $gte / $lt / $lte (#14080)#14425
os-musk merged 3 commits into
mainfrom
claude/issue-14080-ordering-null-comparand-refusal

Conversation

@os-musk

@os-musk os-musk commented Sep 2, 2026

Copy link
Copy Markdown
Collaborator

Fixes #14080

Executes the director seat's ruling A on #14080 (maintainer batch #24, 2026-09-01, 「同意」): a null comparand of $gt / $gte / $lt / $lte — the last null-comparand position the filter contract neither ruled on ($eq: null / $ne: null ARE the null predicate, #5332) nor refused (the 2026-08-31 list-position refusal, #13357 / PR #13673) — is now refused at the same validation entrance as PR #13673, with the same envelope (INVALID_FILTER / 400) and a message that prescribes the ruled spellings. driver-memory's two-face divergence on the shape becomes constructively unreachable; ⛔ the matcher is not repaired (ruling point 4), ⛔ no ordering-vs-null semantics is defined anywhere (ruling point 3), ⛔ nothing under #14079 is touched (ruling point 6). Template: git show e398863ce (PR #13673) — same door, message shape, envelope, pin layout and changeset convention.

Clause-②: yes — the PR carries needs:contract-review at open and stays a draft until the contract review at tier passes (ruling point 5). Self-reading from the diff, below.

Deliverable 1 — the census (gated everything else)

Method: my own — neither PR #13673's body/comments nor the #13357 thread records the 2026-08-31 census method (both were read from the public pages; the only mention is triage's instruction that the executor run one). Script: a Python walker over examples/**, apps/**, packages/**, content/docs/**, skills/** (excluding node_modules, dist, .turbo, .cache, coverage, build), three authoring forms, every spelling the contract accepts:

  • object form { f: { $gt: null } } — regex over TS/TSX/JS/MJS/CJS/MDX/MD, parsed walk over JSON and YAML (so YAML ~ and an empty value count as null);
  • array/tuple form ['f', OP, null] — all 20 AST_OPERATOR_MAP spellings that lower to the four operators (the greater-than sign, gt, greater_than, greaterthan, after; the greater-or-equal sign, gte, greater_than_or_equal, greaterthanorequal, greaterorequal; the less-than sign, lt, less_than, lessthan, before; the less-or-equal sign, lte, less_than_or_equal, lessthanorequal, lessorequal — the four sign spellings are written in words here because GitHub's body sanitizer eats angle-bracket-shaped fragments), regex + parsed walk;
  • view-filter rule form { field, operator: OP, value: null } — every VIEW_FILTER_OPERATORS member and VIEW_FILTER_OPERATOR_ALIASES key that lowers to the four, regex over flat object literals + parsed walk.

Run on the worktree at 8d3f0939d (before any edit); 6,105 text + 361 JSON + 2 YAML files scanned.

operator object form array/tuple form view-filter rule form
$gt 9 raw 0 0
$gte 1 raw 0 0
$lt 0 0 0
$lte 1 raw 0 0

Positive control (fires): four scratch files planted under packages/spec/src/data/__census_control__/ (control.ts, control.json, control.yaml, control.mdx) carrying every form and every operator → the same run counted 28 control hits on top of the same 11 ($gt 4/3/2 · $gte 2/2/1 · $lt 3/3/2 · $lte 2/2/2 as object/array/viewrule, across all four file types); removed → back to 11, git status clean.

Every one of the 11 raw hits, classified (none is an authored filter in examples/**, apps/**, content/docs/**, skills/** or any non-test source):

# where what it is
1-4 packages/services/service-analytics/CHANGELOG.md lines 2249, 2405, 6510, 6606 release-note prose quoting {amount: {$gt: null}} and #5332's "positions no ruling covers" sentence
5 service-analytics/src/strategies/filter-normalizer.ts:198 a comment
6 driver-memory/src/memory-matcher.ts:342 a comment (names the shape as "the one null-comparand position the contract still ACCEPTS" — stale once this lands; see the follow-up finding)
7 driver-memory/src/memory-matcher-null-value-and-comparand.test.ts:63 the file's header comment (same sentence; same follow-up)
8-9 service-analytics/src/__tests__/filter-normalizer-undefined-comparand.test.ts:241-242 a test FIXTURE pinning how the analytics normalizer compiles { d: { $gt: null } } (binds NULL, #5526)
10 service-analytics/src/__tests__/filter-value-type-fidelity.test.ts:470 a test FIXTURE, same consumer, { code: { $gt: null } } through NativeSQLStrategy
11 packages/formula/src/matches-filter.test.ts:184 a test FIXTURE pinning matchesFilter fail-closed on { $lte: null } as never

Reading: 0 authored occurrences ⇒ proceed to deliverable 2 (ruling point 1). ⚠️ Declared judgement, not a mechanical number: hits 8-11 are test fixtures that exercise the degenerate shape's HANDLING in two consumers that sit outside the refused door — the analytics normalizer takes the object-form where without parseFilterAST (lowerAnalyticsWhere lowers only the array form through it) and formula's matchesFilter never calls it — and they stay green on this branch (measured: the three files are untouched and their consumers' inputs do not pass the door). They are pins of a degenerate input, not authors reaching for the shape (the ruling's p1 trigger is 「真实 authored occurrence」/「那时它不再是退化输入」). If the seat reads a test fixture as an authored occurrence, the census is 4 and the card returns to the decision box per ruling point 1 — this PR is a draft and can be closed at no cost. Re-run after implementation with the new pins excluded: the same 11 plus 3 comment lines in the edited door's own docblock; supplementary scan of docs/**, .claude/**, .changeset/**, scripts/**: 3 hits, all prose (two in changesets, one of them this PR's own).

Boundary: the census is of LITERAL null; dynamic producers were not censused beyond the three first-party producers the schema docblock names (lifecycle-service, plugin-email outbox sweep, plugin-auth's adapter) — all three emit .toISOString() strings or the producer's own typed value, never a literal null.

Layer measurement — where the ACCEPT lived (before editing)

packages/spec at 8d3f0939d, calling each layer directly on { n: { OP: null } }:

layer $gt $gte $lt $lte $eq $ne
parseFilterAST (object form) ACCEPT ACCEPT (ruled)
parseFilterAST (array form, all 10 probed spellings) ACCEPT
assertListComparandShapes (runtime shape door) passes — walks only $in / $nin / $between passes
normalizeFilterComparandTypes (runtime type door) passes — null is an accepted comparand TYPE passes
FieldOperatorsSchema / ComparisonOperatorSchema (schema door) REFUSE, zod's generic invalid_union "Invalid input" ACCEPT
FilterConditionSchema ACCEPT (loose record) ACCEPT

So the accept was served by the RUNTIME door: the shape door did not cover the ordering positions and the type door admits null as a type; parseFilterAST never runs the zod schema. The refusal therefore goes where #13673's went — assertListComparandShapes in filter-comparand-shape.ts, which parseFilterAST and the engine seam (@objectstack/objectql's delegating wrapper) both run — and the schema door gets the pointed message by the same replace-only error mechanism as the $between endpoint.

Deliverable 2 — per position, before → after

filter before (8d3f0939d) after (this branch)
{n: {$gt: null}} ACCEPTED REFUSED INVALID_FILTER / 400
{n: {$gte: null}} ACCEPTED REFUSED INVALID_FILTER / 400
{n: {$lt: null}} ACCEPTED REFUSED INVALID_FILTER / 400
{n: {$lte: null}} ACCEPTED REFUSED INVALID_FILTER / 400
['n', OP, null] for all 20 spellings ACCEPTED REFUSED INVALID_FILTER / 400, message names the spelling
{n: {$eq: null}} / {n: {$ne: null}} / {n: null} ACCEPTED ACCEPTED (control, #5332)
{n: {$gt: 0}}, {n: {$gte: ''}}, {at: {$lt: '2026-07-01'}}, {at: {$gt: Date}}, {a: {$lte: {$field: 'b'}}} ACCEPTED ACCEPTED (controls)
{n: {$gt: undefined}} REFUSED by the TYPE door REFUSED by the TYPE door, same sentence (strictly null here)
{n: {$in: [null]}}, $between null bounds REFUSED (#13673) REFUSED (unchanged)
schema door, {$gt: null} REFUSED, "Invalid input" REFUSED, pointed message naming $eq: null / $ne: null

Runtime message (engine-prefixed length pinned under the 500-char client bound; 475 with find('deal'): and close_date):

Operator "$gte" on field "close_date" does not accept a null comparand (at where.close_date.$gte). null is not ordered; no two evaluation faces agree on what it matches. State absence with the null predicate: {"$eq": null} is "has no value", {"$ne": null} is "has a value". Authoring spellings: >=, gte, greater_than_or_equal, greaterthanorequal, greaterorequal. The filter was NOT applied, and an unapplied filter would have returned the UNFILTERED result set.

Files:

  • packages/spec/src/data/filter-comparand-shape.tsORDERING_COMPARAND_OPERATORS (four operators, 20 spellings), nullOrderingComparandError, the strict-null branch in assertFieldListComparands, module note section "Refused BY RULING, 2026-09-01".
  • packages/spec/src/data/filter.zod.tsnullOrderingComparandMessage, one shared orderingComparandSchema(op, label) factory now building BOTH the documentation copy (ComparisonOperatorSchema) and the enforced copy (FieldOperatorsSchema) so they cannot drift (the pairing setMembershipSchema gives the set slots); ORDERING_COMPARAND_DESCRIPTION gains the null sentence.
  • Pins: filter-comparand-shape.test.ts (refusal on both input forms, message contract, nested paths, strict-null controls, the 500-char bound, a reconciliation pin that every ordering spelling in VALID_AST_OPERATORS is refused AND named), filter.test.ts (schema door, both copies, controls), packages/drivers/driver-memory/src/memory-null-ordering-comparand-unreachable.test.ts (mirrors memory-null-list-member-unreachable.test.ts: compile-then-match aborts at the compile face on BOTH readings of "no value" over the card's numeric fixture; positive control; $eq: null control), packages/objectql/src/engine-filter-array-lowering.test.ts (the [finding] driver-memory's matcher answers a NULL comparand inconsistently across the two readings of "no value" — $in:[null] / $nin:[null] disagree while $null / $ne:null agree #13357 block's shape: every verb refuses before any driver call, zero reads/writes, positive control reaches the driver, $ne: null reaches the driver, nested path with the engine prefix). The objectql extension was added because feat(spec): refuse null members in list-comparand positions — $in/$nin members, $between bounds (#13357) #13673's shape included the engine half — declared here as required by the template.
  • .changeset/null-ordering-comparand-refused.md@objectstack/spec: minor (precedent feat(spec): refuse null members in list-comparand positions — $in/$nin members, $between bounds (#13357) #13673; BREAKING accept-set narrowing under the launch-window convention; adr-0087 not-required note).

Controls that stayed green

Ablation — predicted before the run: red on the new pins, green on every control

Committed first (33bfec606), then mutated by script with trap '…restore…' EXIT INT TERM, absolute paths from git rev-parse --show-toplevel:

  • Mutation, proved on disk by anchored counts: door anchor if (ORDERING_COMPARAND_OPERATORS.has(op)) { 1 → 0; zod anchor issue.input === null ? nullOrderingComparandMessage(op) : undefined 1 → 0; git diff --stat: filter-comparand-shape.ts | 6 ------, filter.zod.ts | 4 +---.
  • Resolution path (the ablation's standing condition): spec's own suites import ./filter.zod / ./filter-comparand-shape relatively; driver-memory's vitest.config.ts aliases @objectstack/spec/DATA-NAMESPACE to packages/spec/src/…/index.ts (the packages/runtime's vitest alias list omits @objectstack/spec/cloud, so any runtime test that loads a compiled artifact dies with ENOTDIR #9457 anchored rule), so both read the mutated SOURCE directly — no build leg exists for these two, and none was needed. The objectql pin resolves @objectstack/spec/data through exports (dist, ledgered in KNOWN_UNALIASED_TEST_IMPORTS) and was NOT ablated at dist level — declared, not hidden.
  • Measured under mutation: spec 15 failed / 173 passed — exactly the 15 new pins (9 it.each refusals, message contract, nested paths, the length bound now including the four ordering shapes, the ordering reconciliation pin, and both schema-door message pins), zero controls; driver-memory 6 failed / 9 passed — exactly the six refusal cases, the positive control, the $eq: null control and the [finding] driver-memory's matcher answers a NULL comparand inconsistently across the two readings of "no value" — $in:[null] / $nin:[null] disagree while $null / $ne:null agree #13357 file green.
  • Restore: git checkout HEAD -- ABS-PATH ABS-PATH; git hash-object of both files equals the HEAD: blob (9c274d23…, 5ee9861d…); git diff HEAD empty; re-run: spec 265/265 (4 files), driver-memory 15/15.

Gate union on the final HEAD 163797738

Derived with node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack --commands on the merged tree (origin/main 8eeca27db merged in as 00803ce85; 57 commands, family identical before and after the merge). Exit codes captured after redirect, never through a pipe; verdict lines quoted from each gate.

gate result
pnpm --filter @objectstack/spec test (whole suite as two vitest shards, --maxWorkers=2) shard 1/2: 226 files, 6134 tests passed · shard 2/2: 225 files, 5995 tests passed — 451 files / 12,129 tests, all green
pnpm --filter @objectstack/spec typecheck (tsc + scripts project + test-typecheck) ✓ — "check:test-typecheck: OK — @objectstack/spec's test layer compiles under packages/spec/tsconfig.test.json; 54 file(s) / 262 error(s) / 146 pinned signature(s) held" (the two edited test files are inside that program)
pnpm --filter @objectstack/spec check:generated ✓ "All 15 generated artifacts are up to date" (after gen:api-surface against the fresh dist — no artifact changed)
pnpm --filter @objectstack/spec check:authorable-surface / check:api-surface / check:docs / 12 more spec check:* all ✓ (16/16 in the family)
pnpm --filter @objectstack/driver-memory test 39 files, 1033 tests passed; typecheck
pnpm --filter @objectstack/objectql exec vitest run src/engine-filter-array-lowering.test.ts (dist closure rebuilt on the merged HEAD) 60 passed; pnpm --filter @objectstack/objectql typecheck
36 root check:* / scripts/check-*.mjs commands of the family 33 ✓ · check-dev-prereqs.mjs exit 1 = its own "The workspace is not built — 1 unmet precondition, not a list of problems" (53 packages have no dist; only the two closures were built here) · check-test-completeness.mjs exit 3 and pm/check-half-states.mjs exit 3 = NOT MEASURED by their own text (need a turbo test log / GitHub access)
check:query-options-erasure ✓ "ratchet holds: 67 unswept non-test site(s) … none new" (a first run on 00803ce85 was red — test surface 236 → 238 from two as any in the new engine block; respelled with the file's own asFilterArrayQuery in 163797738)
check:driver-conformance, check:type-check-coverage, check:type-source-resolution
check:dual-build-cjs-loads, check:type-check-debt exit 3 = "PREREQUISITE NOT MET — this gate reads built output, and some package has no dist/" (partial build; NOT a pass, nothing measured)
pnpm lint (eslint . --no-inline-config, whole repo, foreground on 163797738) ✓ exit 0 — and, independently, the narrowed run over the six edited files with --format json: 6 files linted, 0 errors, 0 warnings (the config is not type-aware — no parserOptions.project — so the narrowing excluded nothing this diff could move)

Docs — a deviation, declared

The dispatch's docs item ("one line in content/docs/references/data/filter.mdx as #13673 did") is not producible mechanically: that generated page renders SetOperator (where #13673's SET_MEMBER_DESCRIPTION change surfaced) but no ComparisonOperator / FieldOperators section, so the new sentence in ORDERING_COMPARAND_DESCRIPTION reaches only the JSON schemas and the describe() text; check:docs is green with zero diff under content/docs/**, and no generated artifact was hand-edited. Hand-written candidates for one sentence, if the seat wants one: the operator tables in content/docs/protocol/objectql/query-syntax.mdx (line ~249) and content/docs/data-modeling/queries.mdx (line ~39); #13673 touched neither, so this PR follows the precedent and leaves them.

Clause ② — self-reading from the diff

Yes. Path limb: packages/spec/src/data/filter.zod.ts and filter-comparand-shape.ts (contract schema + the validation door). Content limb: a narrowing of the published filter accept set — { f: { $gt|$gte|$lt|$lte: null } }, in the object form and in every array/authoring spelling, moves from ACCEPTED to REFUSED at parseFilterAST and at every engine verb; nothing widens, no key is added or removed, $eq: null / $ne: null and every non-null comparand keep their verdicts, and the schema door's verdict for null is unchanged (it never parsed) — only its message changes. Semver: @objectstack/spec: minor per the #13673 precedent (BREAKING narrowing under the launch-window convention). Derived judgements for the reviewer: (1) the refusal is strictly nullundefined deliberately keeps the TYPE door's sentence; (2) the enforced and documentation copies are now one factory, which ADDS describe() text to FieldOperatorsSchema's four ordering slots (description only; the union is byte-identical); (3) the door keeps its exported name assertListComparandShapes although it now also refuses two null-shaped carve-outs — renaming would touch the engine's delegating wrapper, which is fenced.

Honest boundaries (not widened, stated)

Serial constraints, re-checked at open

All 21 open PRs' heads fetched (refs/pull/N/head) and diffed against their merge base: none touches any file in this branch's list (filter.zod.ts, filter-comparand-shape.ts, the four test files, the changeset); #14404 and #14381 touch packages/spec/api-surface/*.json, which this branch does not change (gen:api-surface reproduced the committed surface byte-for-byte).

Generated by Claude Code

🤖 Generated with Claude Code

https://claude.ai/code/session_0112hMx9hjJ9BgB28X97DS68


Generated by Claude Code

…te/$lt/$lte)

Implements the 2026-09-01 maintainer ruling (option A) on the last
null-comparand position the filter contract neither ruled on nor refused:
`{ f: { $gt: null } }` and its three siblings are refused at the same
validation entrance as the 2026-08-31 list-position refusal — the runtime
door (`assertListComparandShapes`, inside `parseFilterAST` and at the engine
seam) with the `INVALID_FILTER` / 400 envelope, and the schema door
(`ComparisonOperatorSchema` / `FieldOperatorsSchema`, now one shared slot
factory) with the pointed message in place of zod's generic union text.
The refusal prescribes the ruled null predicates, `$eq: null` / `$ne: null`.
driver-memory's two-face divergence on the shape becomes constructively
unreachable; the matcher is not repaired and no ordering-vs-null semantics
is defined anywhere. Negative pins on the compile face, the engine seam
(every verb, zero driver calls) and the matcher pipeline; the carve-out is
strictly null — every value, `{ $field }`, `$eq` / `$ne` and `undefined`
keep the answers they had.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0112hMx9hjJ9BgB28X97DS68
…ArrayQuery in the #14080 pin

The query-options-erasure ratchet counts a bare `as any` on a query-options
position; the file's own helper names the contract being bypassed and is the
spelling the ratchet asks for.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0112hMx9hjJ9BgB28X97DS68
@github-actions github-actions Bot added the size/l label Sep 2, 2026
@github-actions

github-actions Bot commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

12 anchor(s) derived from 1 changed package(s); no hand-written page names any of them, so this run has nothing to listnot a clean bill of health. This check sees only pages that NAME a derived anchor: one that documents this change in prose, or enumerates it in an authoring dialect, names none and stays invisible to it on every run.

What this run could not see
  • the SDK route bridge reached 47 of 219 client-bound route-ledger rows — the other 172 have no registrar path: tail to select them, so pages documenting THEIR client methods cannot appear above, on this or any run. Of those 172: 14 are remediable by widening that discovery convention (an in-repo file declares the path; the convention did not scan it); 56 are structural — on a ledger where NOT ONE row is declared in-repo, so no discovery change reaches them at any price; 102 are undecided (no in-repo declaration, on a ledger that has other in-repo registrars — absence and an unreadable spelling are not distinguishable here). The rows themselves: node scripts/docs-audit/affected-docs.mjs --bridge-coverage
  • a page that states a rule by its inputs shares no identifier with the emitter that implements the rule, so an emitter-only diff cannot list it — not on this run and not on any run. Measured on fix(driver-sql): emit varchar(maxLength) for a text field a declared index keys on #11430: content/docs/protocol/objectql/types.mdx documents the text-family column mapping by the ObjectQL type names it maps FROM (text / textarea / html) while the diff changed createColumn; it went unlisted, and it was the page that diff falsified, in four places. No shared token exists to detect this on, so a rule your change carries has to be re-read by hand in the pages that restate it.

Coarse fallback — 128 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json 793065de2c03936d4dd88f7026a1530d4c52c462packageMentionDocs.

Which tree this was computed on

This run read content/docs from 19c90d2ee02310d5ed1f9d5609c835362e39a5b5 — the merge of head 16379773843fbd97f1d3dc1e3b2c21a78bba1b82 into base 793065de2c03936d4dd88f7026a1530d4c52c462, which is what actions/checkout gives a pull_request run. Not the PR head.

A worktree cut from an older main holds a different content/docs, so re-deriving there can legitimately return a different list — that is a different tree, not a wrong row. To answer on the same tree:

# while this PR is open — GitHub drops the merge commit once it closes
git fetch origin 19c90d2ee02310d5ed1f9d5609c835362e39a5b5 && git checkout 19c90d2ee02310d5ed1f9d5609c835362e39a5b5
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin 793065de2c03936d4dd88f7026a1530d4c52c462 16379773843fbd97f1d3dc1e3b2c21a78bba1b82 && git checkout -B drift-repro 793065de2c03936d4dd88f7026a1530d4c52c462 && git merge --no-ff 16379773843fbd97f1d3dc1e3b2c21a78bba1b82

node scripts/docs-audit/affected-docs.mjs --json 793065de2c03936d4dd88f7026a1530d4c52c462

⚠️ That checkout carried uncommitted changes, so the commit above does not fully identify what was read.

os-musk commented Sep 2, 2026

Copy link
Copy Markdown
Collaborator Author

Contract review PASS at tier — needs:contract-review cleared on this PR and on #14080 in the same stroke (domain:engine seat, session session_0112hMx9hjJ9BgB28X97DS68, 06:27Z). Provenance: the in-seat review 5505435356 on the card (tier fuse claude-fable-5-1; independent isolated-fable second opinion adopted verbatim, transcript-verified 69/69 messages at tier; dev's open question ruled A — census 0). Governed-surface test on the exact seven-file list: NOT governed. Flip to ready + auto-merge (squash) follows when every check on 163797738 is green (Lint & Repo Gates and Test Core (1/6) still running at 06:27Z).


Generated by Claude Code

@os-musk
os-musk marked this pull request as ready for review September 2, 2026 06:44
@os-musk
os-musk enabled auto-merge September 2, 2026 06:45

os-musk commented Sep 2, 2026

Copy link
Copy Markdown
Collaborator Author

Provenance (domain:engine seat, session session_0112hMx9hjJ9BgB28X97DS68, 06:46Z): flipped to ready and auto-merge (squash) armed on head 163797738. Contract review PASS at tier on the card: 5505435356 (tier fuse claude-fable-5-1; isolated fable second opinion adopted verbatim after transcript verification; the dev's open question ruled A). Both needs:contract-review carriers cleared and read back (5505439772). Every check run on this head completed success or skipped (Lint & Repo Gates 06:34:11Z, Test Core (1/6) 06:35:18Z); mergeable_state: clean; governed-surface test on the exact seven-file list: NOT governed. Landing to-do at MERGED: verify by content on origin/main, strip pm:dispatched from #14080 (Fixes closes it), landing record; #14426 (prose follow-up, Blocked-by: #14425) unblocks.


Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation protocol:data size/l tests tooling

Projects

None yet

2 participants