fix(objectql)!: refuse an engine where that is not a filter before any driver call — a string, number or Map where on a multi-row update or delete rewrote or removed every row - #20144
Conversation
…m, before any driver call A string, number, boolean, `Map`, `Date` or any other value that is neither absent, a filter object nor a filter array is now refused at the top of `lowerWhereFilterArray` with `INVALID_FILTER` / 400, in the wire door's words. The non-filter-array branch carries the same envelope. Claude-Session: https://claude.ai/code/session_01Bvd69VPa6puiNzzPUroDBx Co-authored-by: Claude <noreply@anthropic.com>
…a real SQL table Claude-Session: https://claude.ai/code/session_01Bvd69VPa6puiNzzPUroDBx Co-authored-by: Claude <noreply@anthropic.com>
…wing Claude-Session: https://claude.ai/code/session_01Bvd69VPa6puiNzzPUroDBx Co-authored-by: Claude <noreply@anthropic.com>
… refusal Claude-Session: https://claude.ai/code/session_01Bvd69VPa6puiNzzPUroDBx Co-authored-by: Claude <noreply@anthropic.com>
…at it does not implement `check:objectql-double-limit` and `check:where-matcher` both named the new double: its `find` ignored the caller's bound, and its matcher read a combinator as a field name. It now applies the bound after the filter, by presence, and throws on a combinator or an operator other than `$gt`. Claude-Session: https://claude.ai/code/session_01Bvd69VPa6puiNzzPUroDBx Co-authored-by: Claude <noreply@anthropic.com>
📓 Docs Drift CheckThis PR changes 1 package(s): 13 hand-written doc(s) NAME something this change touched and may need an implementation-accuracy re-verification:
⛔ 4 release-owned page(s) also name something this change touched. These are read-only:
What this run could not see
Coarse fallback — 17 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): Which tree this was computed onThis run read A worktree cut from an older # while this PR is open — GitHub drops the merge commit once it closes
git fetch origin b38aad98721650728760092cffe64646b1bc48d0 && git checkout b38aad98721650728760092cffe64646b1bc48d0
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin f09d4122bc8c2eb39414b83b1924217a04b66cb2 2bd379d22a84715c722a45510d0a9255fa9b3d16 && git checkout -B drift-repro f09d4122bc8c2eb39414b83b1924217a04b66cb2 && git merge --no-ff 2bd379d22a84715c722a45510d0a9255fa9b3d16
node scripts/docs-audit/affected-docs.mjs --json f09d4122bc8c2eb39414b83b1924217a04b66cb2
|
Contract reviewServed-tier: Scope: 4 files (+561/−2) on merge base
Measured in detached worktrees at head and base, through a real ① Derived judgments
② Semver level
③ Boundary flags
Implemented-by: VERDICT: FAIL Must-change (prose only;
|
…ept condition The empty string was not refused under row-level security: the security middleware's composition dropped it as absent. And a filter object is accepted by its built-in tag, so one that overrides Symbol.toStringTag is refused. Prose only. Claude-Session: https://claude.ai/code/session_01Bvd69VPa6puiNzzPUroDBx Co-authored-by: Claude <noreply@anthropic.com>
Contract reviewServed-tier: Scope: a DELTA review of PR #20144 after FAIL 5831634153 on ① Derived judgments
② Semver levelUnchanged:
③ Boundary flags
Implemented-by: VERDICT: PASS |
…the query, not the data — row-independent walk, shape and type doors, unknown having keys, temporal addDays pairs (objectstack-ai#20147) Fixes objectstack-ai#20122 Fixes objectstack-ai#20123 Fixes objectstack-ai#20127 Clause-②: no (narrowing) A refusal on `engine.aggregate` is now a property of the query, never of the data. The per-aggregation `filter` and `having` are judged once, before any driver is asked for a row, and an empty table refuses what a full one refuses. This is the combined claim of objectstack-ai#20122 (the chain head), objectstack-ai#20123 and objectstack-ai#20127, with the seat's amendment 5832109186 folding the per-aggregation filter's shape door into objectstack-ai#20122, and amendment 2 (5832826252) ruling that an array filter is refused too. | commit | card | change | |:--|:--|:--| | `866212209f` | objectstack-ai#20122 | the per-aggregation `filter` loop runs the comparand-TYPE door and a row-independent walk (`assertAggregationFilterIsEvaluable`) | | `71e3ae2092` | objectstack-ai#20123 | a `having` key naming no column of the aggregated row is refused | | `1690b79b59` | objectstack-ai#20127 | `having` evaluates `addDays` only between two temporal columns of one class; each aggregated column's class is read statically (`aggregatedRowColumnClasses`) | | `3c2fa822db` | | merge of `main` at `949e99bed9`, where PR objectstack-ai#20144 landed | | `a17770aa77` | objectstack-ai#20122 | the shape gate `where` takes, on the per-aggregation `filter` (seat amendment 5832109186) | | `796b06a12e` | | changeset prose only | | `211cfba477` | objectstack-ai#20122 | an array per-aggregation `filter`, `[]` included, is refused (seat amendment 2, 5832826252, ruling A) | Session `session_01Bvd69VPa6puiNzzPUroDBx`, branch `claude/issue-20122-aggregate-filter-doors`. The three card commits sit on base `f09d4122bc`. The branch merged `main` at `949e99bed9`, and head is `211cfba477` (`engine.ts` `45301fa9e4`, `having-filter.ts` `876293a1e9`). Readings below name the commit they were taken on. ## 1. Measured first A scratch harness (not committed) ran each shape through the public `ObjectQL.aggregate` and through `POST /api/v1/data/:object/query` (`RestServer`, then `ObjectStackProtocolImplementation.findData`, then `ObjectQL.aggregate`). It used a real `InMemoryDriver` and a real `SqlDriver` (better-sqlite3 `:memory:`), on a populated table (6 rows, groups c1, c2, c3) and on an empty one, and counted driver calls. Per-aggregation filters ran grouped and ungrouped. Each `having` ran on both `applyHaving` doors: the native `driver.aggregate()` door, and the fallback forced by an extra per-aggregation filter. That is 1464 cells per tree, plus 72 for six extra shape-door shapes. The base is `f09d4122bc`, and the shape-door base is the merged tree before `a17770aa77`. On every row below, both drivers and both doors agreed. ### H1 (objectstack-ai#20122): held for the walker's refusals, partly falsified for the type door | `aggregations[i].filter` | base, empty table | base, populated | head | |:--|:--|:--|:--| | `{ amount: { $median: 1 } }`, `$nand`, `$regex`, `$regex` with `$options`, `$like`, a dangling `$like` escape, `$ilike`, a non-`$` key beside an operator, `$median` under `$not`, a bare `{ $field }` | `200 []` grouped, `[{ n: 0 }]` ungrouped | 400 `INVALID_FILTER`, raised after `find` (1 call) | 400 `INVALID_FILTER`, 0 driver calls, both populations, engine and REST | | an empty or non-string `$icontains` | the same at the engine; the REST door already refused it (`VALIDATION_FAILED` / 400), whatever the rows | the same | 400 at the engine, 0 calls; REST unchanged | | `{ nope: { $median: 1 } }` (a column the source row does not carry) | `[]` | counted no row in any group, no error | 400, 0 calls | | `$median` in a `$or` branch after one that held | `[]` | counted EVERY row (c1 2, c2 3, c3 1) | 400, 0 calls | | a `{ $field }` as an `$in` member or `$contains` pattern / as a `$nin` member or `$exists` operand | `[]` | no row / every row | 400, 0 calls | | `addDays: 1.5` / `addDays: '7'` | `[]` | answered (c1 1, c2 2, c3 1 / c1 2, c2 2, c3 1) | 400, 0 calls | | type door: `{ $eq: { v: 1 } }`, an implicit `undefined`, `{ $eq: new Map() }`, a function under `$gt`, an `undefined` `$in` member, a bigint beyond 2^53, `{ $gt: { $field: 5 } }` | `[]` | counted no row | 400, the type door's words rooted at `aggregations[i].filter`, 0 calls | | type door: a `Symbol` under `$ne` / under `$gt` | `[]` | every row / a raw `TypeError` with no `code` and no `status` | 400, 0 calls | So H1 holds for every walker refusal. For the type-mismatched comparands the premise was only half right. They were not refused on a populated table either: they answered silently, apart from the uncoded `Symbol` throw. The head refuses all of them before any read, as H1's head asks. The exact-range bigint changes an answer instead of narrowing. `{ amount: { $in: [400n, 20n] } }` counted no row, and it now counts c1 1, c3 1, because it is narrowed as `where` narrows it. ### H2 (objectstack-ai#20123): held | `having` | base, both populations | head | |:--|:--|:--| | `{ totl: { $gt: 100 } }`, `{ totl: 500 }`, `{ amount: { $gt: 100 } }` (a source column), `{ $and: [{ total: { $gt: 0 } }, { totl: … }] }`, `{ 'customer_id.name': 'x' }`, `{ customer_id: 'c1' }` under an aliased groupBy | no group, no error | 400 `INVALID_FILTER`, 0 driver calls, both doors, engine and REST | | `{ totl: { $ne: 1 } }`, `{ totl: { $exists: false } }`, `{ $not: { totl: … } }`, `{ $or: [{ total: { $gt: 0 } }, { totl: … }] }` | EVERY group on a populated set | 400, 0 calls | Controls, accepted and byte-identical: a groupBy column, an aggregation alias, a `count` alias, a structured item's alias (`cust`), keys nested under `$and` / `$or` / `$not`. ### H3 (objectstack-ai#20127): held | `having` pair with `addDays` | base, populated | head | |:--|:--|:--| | two numeric aliases (`total` vs `max_cap`) | no group | 400, "addDays adds whole days to a date or datetime column, and "max_cap" is numeric — an offset has no meaning on it." | | a `count` against itself, `addDays: 0` | every group | 400, same words | | `date` vs numeric, numeric vs `date`, groupBy text vs `date` | no group | 400, driver-sql's cross-class sentence | | `date` vs `datetime` / `datetime` vs `date` | c3 / c2, c3 | 400, cross-class | | a `date` pair whose offset column is text or `date` | no group | 400, "the addDays offset … is not a numeric column, and a day offset must be a number of days." | Controls, byte-identical: `date` / `date` with `7`, with `-3`, with an offset from a numeric `max`, and with an offset from a `count`; `datetime` / `datetime`; a `day` bucket against a `date`; and every `{ $field }` pair WITHOUT `addDays`. ### H4: held 664 control cells over 43 control shapes answered byte-identically at base and head. They cover the objectstack-ai#20122 controls (implicit equality, `$gt`, `$in`, `$nin`, `$between`, `$icontains`, `$startsWith`, `$ne: null`, `$exists`, `$null`, `$or`, `$not`, `{}`, a scalar `{ $field }`, `addDays` date pairs, an exact bigint, a `Date` bound), the objectstack-ai#20123 and objectstack-ai#20127 controls above, the structured-groupBy probes, and the H5 shapes the shape gate leaves alone. ### H5: folded in (seat amendments 5832109186 and 5832826252) | `aggregations[i].filter` | base, engine | base, REST | head, engine | |:--|:--|:--|:--| | a string | memory: every row of every group; sql: `NOT_IMPLEMENTED` / 501 (driver-sql's native aggregate saw the key) | `VALIDATION_FAILED` / 400 | 400 `INVALID_FILTER`, 0 driver calls | | a number, `true`, `false`, `0`, `''` | every row of every group, both drivers | `VALIDATION_FAILED` / 400 | 400, 0 calls | | a `Map`, a `Date`, a `Set` | every row of every group | (not JSON) | 400, 0 calls | | `[]` | no filter (every row) | `VALIDATION_FAILED` / 400 | 400 `INVALID_FILTER`, 0 driver calls (from `211cfba477`) | | `[['amount', '>', 100]]` | counted no row: the walker reads its index keys as column names | `VALIDATION_FAILED` / 400 | 400 `INVALID_FILTER`, 0 driver calls (from `211cfba477`) | | a null-prototype filter object | filters correctly | (not JSON) | unchanged | The reach is in-process only: the REST door refuses every JSON shape through `AggregationNodeSchema`. An array is refused because the slot is declared `FilterConditionSchema`, which admits no array form (the condition-array sugar is lowered on `where` alone), and REST already refused every array there with `VALIDATION_FAILED`: one rule at both doors, as `having`'s condition check has on objectstack-ai#20099. ## 2. What changed - **`packages/objectql/src/engine.ts`, `ObjectQL.aggregate` only.** - The per-aggregation `filter` loop now opens with the shape gate `where` takes. It reuses `isWhereFilterObject` / `describeNonFilterWhere` from PR objectstack-ai#20144 and `where`'s words, adapted because no array is accepted here: "`aggregate('order'): 'aggregations[1].filter' must be a filter object, received string "…". It was not applied, and an unapplied filter would have aggregated every row of each group for that aggregation.`". An array of any length, `[]` included, is refused first, naming the object form: "`… must be a filter object, received an array ([["amount",">",100]]). The condition-array form … is input-only sugar lowered on 'where' alone …`". `undefined`, `null`, a plain object and a null-prototype object pass as before. - After the doors the loop already ran, it adds `normalizeFilterComparandTypes` rooted at `aggregations[i].filter`, copying a narrowed bigint on write, and then `assertAggregationFilterIsEvaluable`. - The `having` entry passes `aggregatedRowColumnClasses(groupBy, aggregations, object fields)` to `assertHavingIsEvaluable`. - **`packages/objectql/src/having-filter.ts`.** - The row-independent walk takes a scope: the clause whose words it speaks (`having`, or `aggregations[i].filter` through `aggregationFilterClause`), plus, where the position has a closed namespace, the column set and the column classes. - `assertAggregationFilterIsEvaluable(filter, index)` runs the walk with the per-aggregation clause and no column set. The filter reads the object's raw columns, whose names the engine does not judge on `where` either, so the `{ $field }` NAME check stays `having`'s. - The objectstack-ai#20123 refusal (`unknownHavingColumnError`) collects keys naming no column of `aggregatedRowColumns` at any depth and refuses them once the rest of the clause has passed. Operators are judged first, so `{ nope: { $median: 1 } }` keeps its operator refusal, and its pin is unchanged. The refusal names every unknown key, the first with its position, and lists the columns. It opens the way the REST ingress's unknown-`where`-field refusal does: "`having` filters on 'totl' at having.totl, which is not a column of the aggregated row … so the query was refused instead of answered." - For objectstack-ai#20127, `aggregatedRowColumnClasses` reads each column's class statically. A groupBy projection takes its field's declared type through the spec's value-class sets. A `day` bucket is a `date`, by the `YYYY-MM-DD` label contract, and a coarser bucket is a text label. `count` / `count_distinct` / `sum` / `avg` are numeric, and `min` / `max` take their field's type. A class the declaration cannot tell (no field map, an undeclared field, a `formula`) is not judged. `assertOffsetPairIsTemporal` applies driver-sql's `addDays` arm in its order, in its sentences: same class, then a temporal referent, then a numeric offset column. - **Tests**: - `engine-aggregate-filter.test.ts` gains the objectstack-ai#20122 tables. Each refusal runs on both stand-in driver kinds, empty and populated, grouped and ungrouped, asserting `code`, `status`, one message and 0 driver calls. The walker rows are held to the per-row floor's words, and the type rows to the type door's own words. The file also gains controls and the shape-gate table. - `engine-aggregate-having-comparand-shape.test.ts` gains the objectstack-ai#20123 and objectstack-ai#20127 tables, both doors, empty and populated, with controls. - **Three changesets**, one per card, all `@objectstack/objectql` `minor`. ## 3. Declaration (H6) `Clause-②: no (narrowing)`, BREAKING, `minor`. The accept set only narrows. The one answer change of an already-accepted input is the bigint narrowing, a correction to the answer `where` gives. - `check-changeset-no-major --base origin/main`: "✓ This diff introduces no `major` bump." Its level axis reads NOT APPLICABLE locally, with no `pull_request` payload; CI reads it from this body. - `check-adr-0087-registration --base origin/main`: "✓ check-adr-0087-registration: 3 declared-breaking changeset(s), each carrying an ADR-0087 disposition." - 20122: `not-required (already-registered filter-between-field-reference-endpoint-refused, filter-icontains-comparand-refused-at-parse, filter-regex-options-retired)`. - 20123 and 20127: `not-required (no-migration-prescription)`. Their tables record before and after and prescribe no rewrite: a typo'd column or a non-temporal `addDays` pair has no accepted spelling to migrate to. ## 4. Tests, reverse verification, ablation At `211cfba477`: - `@objectstack/objectql`, whole suite, both vitest projects: `Test Files 317 passed (317)` · `Tests 5589 passed (5589)`. The two aggregate files alone: `Test Files 2 passed (2)` · `Tests 211 passed (211)`. - `pnpm --filter @objectstack/objectql run typecheck`: exit 0. `check:test-typecheck` reads "OK … 40 file(s) / 234 error(s) / 65 pinned signature(s)", unchanged. At `a17770aa77` (before the array refusal; not re-run for `211cfba477`): - Consumer suites, run against the rebuilt `objectql` `dist/`, which carries the new symbols (2 hits each): - REST (`list-view-grouping-query-door`, `rest-server-canonical-query-ast`, `request-schema-gate.conformance`): 81 passed, 1 skipped (pre-existing); - `metadata-protocol` (`protocol.query-param-arity`, `protocol.read-verb-canonical-fold`): 64 passed; - `plugin-security` `predicate-guard`: 10 passed; - the dogfood aggregate and analytics tests (`analytics-inline-dataset-admission`, `analytics-label-scope`, `analytics-rls`, `analytics-timezone`, `date-bucket-parity-conformance`, `date-bucket-parity-turso`, `empty-group-bucket-parity`, `group-key-read-shape-parity`) plus PR objectstack-ai#20144's `engine-where-shape-refusal`: `Test Files 9 passed` · `Tests 55 passed`. - **Reverse verification.** `engine.ts` and `having-filter.ts` were restored to their merge-base blobs (`4f5d28170b`, `7d9f8ace64`) under an EXIT/INT/TERM trap, with the fix committed first. The two test files then read `59 failed | 150 passed (209)`, which is exactly the new refusal and narrowing rows. The restore was proven by HEAD-blob equality and an empty `git diff HEAD`. At `1690b79b59`, before the shape gate, the same run against the `f09d4122bc` blobs read `53 failed | 148 passed (201)`. - **Ablation**, one per door, through `scripts/ablation-replace.mjs`: the first four at `1690b79b59`, the shape gate at `a17770aa77` and again, branch by branch, at `211cfba477`. Each anchor hit once (x1 to x0), the blob moved, and the file was restored to its HEAD blob with `git diff HEAD` empty. The tests import `./engine.js` from source, so no `dist/` is on their resolution path. | ablated | failed / total | red set | |:--|:--|:--| | `assertAggregationFilterIsEvaluable(typed, i)` | 19 / 201 | the 12 walker rows and the 7 reference rows | | the type-door call (the filter passed as is) | 11 / 201 | the 10 type rows and the bigint narrowing | | the objectstack-ai#20123 key push | 12 / 201 | the 10 unknown-key rows, the "every key named" row, the aliased-groupBy row | | the objectstack-ai#20127 `assertOffsetPairIsTemporal` call | 11 / 201 | the 10 pair rows and the month-bucket row | | the shape gate's condition (to `false`), at `a17770aa77` | 6 / 209 | the 6 shape rows | | at `211cfba477`: the array branch's condition (to `false`) | 3 / 211 | the 3 array rows | | at `211cfba477`: the non-object branch's condition (to `false`) | 6 / 211 | the 6 non-object rows | | at `211cfba477`: both conditions (the whole gate) | 9 / 211 | the 6 non-object and 3 array rows | ## 5. Gates - Derived with `node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack`, 7 paths against merge base `949e99bed`: 64 commands. Each was run before any pipe and its exit code recorded, at `796b06a12e`: all 64 exit 0. At `211cfba477` the gates the patch round touches were re-run: - `check-changeset-no-major --base origin/main --event` (this body): "✓ This diff introduces no `major` bump." · "✓ LEVEL AXIS: this PR declares clause-② `no (narrowing)`, and no package whose `packages/**/src/**` it moves is graded `patch`." - `check-adr-0087-registration --base origin/main`: "✓ check-adr-0087-registration: 3 declared-breaking changeset(s), each carrying an ADR-0087 disposition." - `check-empty-changeset --base origin/main`: "✓ No empty-frontmatter changeset introduced by this diff (3 declaring changeset(s) added)." - `check-issue-citations --base 949e99b`: "every citation this change adds resolves" (25 judged). - `check:nul-bytes`: "OK (scanned 9571 text file(s) … no raw ASCII control bytes)". `--ran`: "Run reconciliation — 64 derived, 64 run, 0 NOT-MEASURED, 0 UNRUN." - `check:dual-build-cjs-loads` first exited 3 (PREREQUISITE NOT MET). The eight packages it named were built, and it reran at exit 0. - `node scripts/check-issue-citations.mjs --base 949e99b`: "every citation this change adds resolves" (24 judged). - Lint, a declared narrowing, because `pnpm lint` is CI's. `eslint --no-inline-config --format json` over the 4 changed `.ts` files reads 4 files, 0 errors, 0 warnings, 0 fatal. `--print-config` returns a config for each file. `eslint.config.mjs` sets no `parserOptions.project` and no typed rule, so no untouched file's verdict can move. - Control-byte self-scan of the 7 changed files: grep exit 1 (none). ## 6. Deviations and conflicts, declared - **The array question is ruled.** The first round kept arrays outside the shape gate, as the first amendment instructed, and held a condition array's old answer (no row) in a test marked as kept, not approved. The seat ruled A in amendment 2 (5832826252): an array, `[]` included, is refused, because `AggregationNodeSchema.filter` is `FilterConditionSchema` and REST already refuses every array. `211cfba477` does that; the held control became three refusal rows. - **objectstack-ai#20123's code.** The card's suggested shape said "the unknown-field refusal's envelope", which is `INVALID_FIELD`. The dispatch's H2 said `INVALID_FILTER`. This PR uses `INVALID_FILTER`, the code of every other `having` refusal, including objectstack-ai#20099's unresolved `{ $field }` refusal over the same column set. The name is a column of the query's own projection, not a field of the object, and the words follow the unknown-`where`-field refusal. - **Real drivers in the pins.** The committed pins use the stand-in drivers with call counters. `@objectstack/objectql` has no driver dependency, and the amendment keeps the file surface unchanged. The `driver-memory` and `SqlDriver` legs, through the engine and through REST, are the scratch measurement above. - **The `{ $field }` refusal words for a bare reference in a per-aggregation filter.** On a populated table this was already refused, as an unsupported `$field` operator. It is now refused whatever the rows, in the walk's bare-reference words, with the same code and status. No committed test pinned the old text. ## Acceptance notes Observed and not fixed here. The report carries each with its class and evidence; the seat files the per-aggregation family as one class-closure card (amendment 2, 5832826252). - **Docs drift check (5832744152), read and receipted here.** It names six hand-written pages, each only through the `count_distinct` literal in `NUMERIC_RESULT_FUNCTIONS`. Re-read against this change: none states anything it falsifies. `data-modeling/queries.mdx` already gives `having`'s namespace as the aggregated row's own columns, and `protocol/objectql/query-syntax.mdx` gives the `addDays` class rule this change now applies to `having`. One pre-existing stale row, not introduced here: `query-syntax.mdx`'s table of members "not executed on the `find()` path" still calls `aggregations[].filter` "EXPERIMENTAL — not enforced", although the engine has enforced it since objectstack-ai#10576. Left to the docs lane. The two release-owned pages are read-only. - The per-aggregation filter still lacks `where`'s temporal-comparand door. `{ placed_on: { $gt: 'not-a-date' } }` counts no row on both drivers, while the same predicate as a `where` is refused 400. - A `Date` comparand in a per-aggregation filter counts no row against an ISO-text `datetime` column on both drivers: the walker compares a string with a `Date`. `driver-sql` answers the rows for the same `where`. - `addDays` on a numeric pair in a per-aggregation filter still answers by coercion (no row). The objectstack-ai#20127 rule classifies only the aggregated row's columns; the per-aggregation position would read its classes from the object's declared types, where `driver-sql` withholds the reason from the wire. - A `{ $field }` in a per-aggregation filter naming no field of the object counts no row, where `driver-sql` refuses it on `where`. The engine keeps its registry-less tolerance on names. - `POST /data/:object/query` with `aggregations: [{ …, filter: { nope: 1 } }]` answers 200 with zero counts. The REST ingress refuses the same unknown key in `where` (`INVALID_FIELD` / 400), and it does not judge per-aggregation filter keys. - A `having` `{ $field }` pair across classes WITHOUT `addDays` (a sum against a date) still answers by coercion. `driver-sql` refuses cross-class pairs on `where`, but `FieldReferenceSchema` declares the class rule only for `addDays`, and this change keeps to that. --------- Co-authored-by: Claude <noreply@anthropic.com>
Fixes #20121
Clause-②: no (narrowing)
Write verbs first — the base reading: a whole-table rewrite and a whole-table delete
Measured on
origin/maina08e059c61before any edit, through the realObjectQL, four rows seeded (two per owner), ondriver-memoryand onSqlDriver(better-sqlite3:memory:). The rows touched were read back past the engine.SecurityPluginSecurityPlugin, system contextSecurityPlugin, RLS-scoped memberupdate(o, {name:'Z'}, { where: 'amount > 100', multi: true })INVALID_FILTER, 0 rewrittenwhere: 42/where: new Map(...)delete(o, { where: 'amount > 100', multi: true })INVALID_FILTER, 0 deletedwhere: 42/where: new Map(...)where: [1, 2, 3]code/statusundefined, 0multiwhere: { amount: { $gt: 100 } }or[['amount','>',100]],multi: trueupdate()anddelete()lower at the[#5158]comment ("Lower before the by-id extraction below readswhere.id" / "Same ordering reason as update()"), thenresolveEngineUpdateDispatch/resolveEngineDeleteDispatchlook for awhere.idonly in an objectwhere, find none, and return{ kind: 'multi' }onoptions.multi. The seeded AST carries the string, and the driver'supdateMany/deleteManyignores it.SecurityPluginand under a system context with it. For an RLS-scoped caller it depends on the value:Map(and aDate,Setortrue) is AND-composed by the security middleware into$and, where the driver's node gate refuses it: 0 rows.security-plugin.ts,opCtx.ast.where ? { $and: … } : …) reads a falsywhereas absent and drops it, so the member's''read all 2 of their rows, andupdate(multi)/delete(multi)rewrote or deleted 2 of 4 on both drivers: every row the member could reach.dispatchUnscopedMultiWriteHooks, thesys_attachment/sys_commentpredicate-less-write refusals) treats only an absent ornullwhereas unscoped, so a string / number /Mapwhereskipped that guard while the driver treated it as no predicate at all.The re-grade to p0 is the seat's act; this PR lands the refusal.
What changed (read from the code at the head of this branch)
packages/objectql/src/engine.ts,lowerWhereFilterArrayonly:wherethat is notundefined, notnull, not an array and not a filter object is refused withinvalidFilterError(the package's existing ADR-0112 builder:code: 'INVALID_FILTER',status: 400,httpStatus: 400). All six verbs (find,findOne,count,aggregate,update,delete) call the seam before they resolve a driver, so the refusal comes before any driver call. The message uses the wire door's words:VERB('OBJECT'): 'where' must be a filter object or condition array, received string "amount > 100". It was not applied, and an unapplied filter would have returned the unfiltered result set.Forupdate/delete, the last clause readsupdated every row in scope/deleted every row in scope.invalidFilterErrorinstead of a bareError.isWhereFilterObject: a non-array object whose built-in tag (Object.prototype.toString, which readsSymbol.toStringTagthrough the prototype chain) is[object Object]. That covers a plain object,Object.create(null), a class instance, aProxyof one, and an object from another realm. An object that overridesSymbol.toStringTag, own or inherited, is refused and named by its tag (H4 below).No new export, no new error code, no change to the drivers, to the REST door, to the
filteralias or to thehavingregion. The docblock hunk draft PR #20125 edits is not touched.H4 — every
whereshape, base answer and head answerMeasured on both drivers with no security;
find/update(multi)/delete(multi)shown (count and aggregate followfind).MapINVALID_FILTER/ 400, 0 driver calls[1,2,3]code/statusundefinedINVALID_FILTER/ 400, 0 driver callsDate,Set,true,''INVALID_FILTER/ 400, 0 driver callsundefined,null{},[]Object.create(null)with the filterSymbol.toStringTag(own or inherited) with the filter on own keysINVALID_FILTER/ 400,received Criteria— the one correctly-answering shape this narrows; no producer in the repo creates onenullstays accepted:findOne's no-predicate guard and the unscoped-write detector both already readnullas absent.''is refused, although the wire door reads a blank?filter=as absent. At the engine,''deleted every row while the unscoped-write guard read it as scoped. Under RLS it was not refused either: the security middleware dropped it as absent, and the member's every reachable row (2 of 4) was rewritten or deleted. The wire door's normalizer deletes a blank?filter=before the engine sees it; other REST paths were not measured for''.findOnekeeps its own no-predicate refusal (nocode) fornull/undefined/{}/[], exactly as at base.H3 — other doors that take a caller
where(census, base and head,driver-memory, bad value'amount > 100', control{ amount: { $gt: 100 } })createContext().object(o).find/.delete({ multi: true })INVALID_FILTER/ 400 (pinned)engine.aggregate(o, { where: filter }))service-analyticsplugin.tsaggregateaggregateaggregateaggregations[].filterbeforeFindhook assigningctx.input.ast.whereopCtx.ast.wherebeforeUpdate/beforeDeletehook assigninginput.options.whereonmulti: trueupdateMany/deleteMany/upsert/distincton the engineupsertanddistinctare retired)Tests (at
aa923a1f5c, the code head, unless noted)Head
2bd379d22achanges only the changeset prose, after contract review 5831634153.engine.ts(4f5d28170b) and both pins are blob-identical toaa923a1f5c, so every test reading below stands for the head.packages/objectql/src/engine-where-shape-refusal.test.ts, 47 cases, all through a counting driver double: 6 verbs × 4 bad shapes, each assertingcode,status, theVERB('deal')prefix, zero driver calls and an unwritten table; 6 × 2 controls; the scoped-repository door; five accepted edge shapes; five refused edge shapes ondelete(multi). Run together withengine-filter-array-lowering.test.ts:Test Files 2 passed (2) · Tests 107 passed (107).packages/qa/dogfood/test/engine-where-shape-refusal.test.ts:update(multi)anddelete(multi)× the 4 bad shapes onSqlDriver(better-sqlite3), reading the table through knex; plus two controls. Against the baseobjectql/dist:8 failed | 2 passed(the 6 string/number/Mapcells: "expected the engine to refuse, and it answered"; the 2 array cells:expected undefined to be 'INVALID_FILTER'). After rebuildingobjectql:10 passed(rerun at the head:10 passed).@objectstack/objectqlwhole suite (both vitest projects), at6485915e34:Test Files 317 passed (317) · Tests 5499 passed (5499). Between that commit and the head, only the new test file changed: one repository case was added, and the double now honourslimitand refuses combinators.engine.tsdid not change.@objectstack/plugin-securitywhole suite, at6485915e34against the rebuiltobjectqldist/:Test Files 135 passed (135) · Tests 2685 passed (2685).where, at6485915e34, a declared subset (CI's Dogfood Regression Gate runs all of them): the new pin,attachments-unscoped-delete-gate,bulk-widener-probe,owner-anchor-and-bulk-writes,authored-row-write-scope,comments-permission-matrix,attachments-permission-matrix,owd-public-read-write-write-floor,hook-runas-fls,showcase-crud-persona-matrix:Test Files 10 passed (10) · Tests 131 passed | 1 skipped (132). The skip is the pre-existingdescribe.skipIf(!organizationsAvailable)inattachments-permission-matrix.pnpm --filter @objectstack/objectql run typecheckgreen, includingcheck:test-typecheck(ledger held, 40 files / 234 errors / 65 signatures); the new test file is in thetsconfig.test.jsonprogram (--listFiles).pnpm --filter @objectstack/dogfood run typecheckgreen.check-changeset-no-major --base a08e059c6: nomajor.check-adr-0087-registration --base a08e059c6: 1 declared-breaking changeset, dispositionnot-required (no-migration-prescription).Ablation, run at
544762f243(the 46-case pin, before the repository case was added), both legs throughscripts/ablation-replace.mjs(anchor hits verified on disk, restore proven by blob equality withHEADand an emptygit diff HEAD), with the fix committed first:if (where !== undefined && … !isWhereFilterObject(where))line →if (false)):23 failed | 23 passed (46). Exactly the 18 string/number/Mapcells plus the 5 refused edge shapes turned red; the 6 array cells stayed green on the array branch. Predicted direction, observed.invalidFilterError(→new Error(:6 failed | 40 passed (46), exactly the six non-filter-array cells.Gates
aa923a1f5cwithnode scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack(4 paths vs merge basea08e059c6): 68 commands, all exit 0, captured before any pipe.--ranreconciliation:68 derived, 68 run, 0 NOT-MEASURED, 0 UNRUN, a zero derived from the recorded exit codes.c51cd3d02bturned up two real reds, both in the new test double:check:objectql-double-limit(the double'sfindignored the caller'slimit) andcheck:where-matcher(its matcher read a combinator as a field name). Both are fixed inaa923a1f5c. The same pass hadcheck:dual-build-cjs-loadsat exit 3,PREREQUISITE NOT MET(eight unrelated packages had nodist/). That is not a red: those eight were built and the gate reran green (105 published require entry point(s) across 67 package(s) load).node scripts/check-issue-citations.mjs --base a08e059c61:every citation this change adds resolves(6 judged).2bd379d22aafter the prose patch:check-changeset-no-major --eventover this body,check-adr-0087-registrationandcheck-issue-citations, all exit 0 (quoted in the patch report).pnpm lint, the full dogfood suite,Build Core, andTemporal Conformance(live PG and MySQL).Acceptance notes
aggregate's per-aggregationfilterdrops a string the same waywheredid. Base and head,driver-memory:aggregations: [{ function: 'count', field: 'id', alias: 'n', filter: 'amount > 100' }]counts every row (control{ amount: { $gt: 100 } }counts one per group).AggregationNodeSchema.filteris declaredFilterConditionSchema. The loop that walks it ([#10576]inaggregate()) is a separate door fromlowerWhereFilterArray, so it is outside this claim. No public door or real producer was measured: the wire door parsesaggregationsthroughAggregationNodeSchema, and the analyticsObjectQLStrategyproduces objects.ctx.input.ast.whereinbeforeFind,opCtx.ast.wherein a middleware) is not re-checked: a string there reads every row (census above). The written value is the hook's, not the caller's, and this seam runs before either.''. For a string, number,Map,Date,Setortrue, the security middleware wrapped the value into$and, where the driver's own node gate answersINVALID_FILTER. That refusal is now unreachable for those shapes, because the engine refuses first. The empty string was dropped as absent by that composition (opCtx.ast.where ? … : …) and reached every row the member could reach; the engine now refuses it too, before the middleware runs.Generated by Claude Code