feat(spec): refuse a null comparand in the ordering positions — $gt / $gte / $lt / $lte (#14080)

.changeset/null-ordering-comparand-refused.md

@@ -0,0 +1,46 @@
+---
+"@objectstack/spec": minor
+---
+
+feat(spec): refuse a `null` comparand in the ordering positions — `$gt` / `$gte` / `$lt` / `$lte` (#14080)
+
+**BREAKING** accept-set narrowing on the filter contract, shipped as `minor`
+under the repo's launch-window convention for breaking changes — the same
+convention, the same door and the same envelope as the 2026-08-31 refusal of
+`null` in the list-comparand positions (`$in` / `$nin` members, `$between`
+bounds). Maintainer ruling 2026-09-01 (option A): the four ordering positions
+were the last null-comparand positions the contract neither ruled on
+(`$eq: null` / `$ne: null` ARE the null predicate) nor refused, and
+`driver-memory`'s two faces answered them differently — the live path reads
+two absences as equal, so `$gte: null` admits the no-value row; the reference
+matcher compares through JS coercion, so `5 > null` is `5 > 0`. The contract
+now refuses the shape loudly at the validation entrance, so that divergence is
+constructively unreachable — ⛔ no ordering-vs-null semantics is defined
+anywhere, ⛔ the matcher is not repaired, ⛔ no cross-backend alignment.
+
+What is refused, and where:
+
+- **Runtime door** (`assertListComparandShapes`, run inside `parseFilterAST`
+  and at the engine seam on every verb): `{ f: { $gt: null } }` and its three
+  siblings, in the object form and in every array/authoring spelling that
+  lowers to them (`>`, `gt`, `greater_than`, `after`, `before`, …), are refused
+  with the platform envelope (`INVALID_FILTER` / 400). Previously the shape
+  reached the backends unexamined.
+- **Schema door** (`ComparisonOperatorSchema` / `FieldOperatorsSchema`): `null`
+  never parsed (the slot is `number | Date | string | { $field }`); it now gets
+  the pointed message instead of zod's generic union text, and the two copies
+  are built from one shared slot factory so they cannot drift.
+
+The refusal text prescribes the ruled spellings: `{"$eq": null}` is "has no
+value", `{"$ne": null}` is "has a value". The carve-out is null-shaped and
+nothing wider: every number, `Date`, string (`''` included) and `{ $field }`
+comparand keeps parsing, `$eq: null` / `$ne: null` are untouched, and
+`undefined` keeps the comparand-TYPE door's own message.
+
+**Migration.** A filter refused by the new check had no portable meaning to
+preserve — the two in-memory faces already disagreed on it. Spell the intent
+explicitly: `{ f: { $eq: null } }` for "has no value", `{ f: { $ne: null } }`
+for "has a value", and `$or: [{ f: { $gte: X } }, { f: { $eq: null } }]` for
+"at or above X OR has no value".
+
+<!-- adr-0087: not-required (no-migration-prescription) A validity narrowing over existing keys: no key is removed, renamed or re-shaped, so there is no tombstone and nothing mechanical for `objectstack migrate meta` to rewrite. The refusal reaches an affected author at the parse/query site carrying the remedy; which explicit spelling matches the author's intent ($eq: null, $ne: null, or $or with one of them) is an authoring decision no migration entry can perform — and the ruling's precondition census measured zero authored occurrences of the refused shape. -->

packages/drivers/driver-memory/src/memory-null-ordering-comparand-unreachable.test.ts

@@ -0,0 +1,122 @@
+// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
+
+/**
+ * [#14080] Ruling point 4's NEGATIVE pin, matcher side: a refused null
+ * ORDERING comparand cannot reach this package's reference matcher.
+ *
+ * # What was ruled (2026-09-01, option A)
+ *
+ * #14080 measured, on this package's two faces and the card's numeric
+ * fixture, that `{n: {$gt: null}}` / `{$gte: null}` / `{$lte: null}` answer
+ * DIFFERENTLY: the live (mingo) path reads two absences as EQUAL, so
+ * `$gte: null` admits the no-value row and `$gt: null` does not, while the
+ * reference matcher compares through JS coercion, so `5 > null` is `5 > 0`.
+ * It was the last null-comparand position the contract neither ruled on
+ * (`$eq: null` / `$ne: null` ARE the null predicate, #5332) nor refused (the
+ * 2026-08-31 ruling refused the `$in` / `$nin` members and the `$between`
+ * bounds, #13357). The ruling REFUSES the shape at the contract's validation
+ * entrance (`@objectstack/spec`, `assertListComparandShapes`, run inside
+ * `parseFilterAST` and at the engine seam) instead of defining the semantics:
+ * the divergence becomes constructively unreachable, ⛔ deliberately not
+ * repaired (「⛔ 不单独修 matcher(死代码)」) and ⛔ no ordering-vs-null rule is
+ * stated anywhere (「B(定义语义)排除」), so NOTHING in this file asserts what
+ * either face would have answered. `memory-matcher-null-value-and-comparand.test.ts`
+ * keeps those cells deliberately absent for the same reason.
+ *
+ * # What this file pins, and its honest boundary
+ *
+ * The same pipeline and the same boundary as
+ * `memory-null-list-member-unreachable.test.ts`: a direct caller of this
+ * driver compiles its filter with `parseFilterAST` and hands the result over,
+ * and this file drives that pipeline end to end, pinning that for every
+ * refused shape it ABORTS at the compile face, on BOTH readings of "no value",
+ * before any row is consulted. The engine half (every verb, driver-call
+ * witness) is pinned in `@objectstack/objectql`'s
+ * `engine-filter-array-lowering.test.ts`; the wire/protocol face runs the same
+ * `parseFilterAST`. `match()` and `InMemoryDriver.find()` remain plain library
+ * functions — a caller that skips the compile face meets only this package's
+ * own `assertFilterConditionShape`, which is deliberately NOT extended to the
+ * null-ordering rule (⛔ 不做跨后端对齐工程). Same boundary as every #5869
+ * refusal since #9228; not widened here.
+ */
+
+import { describe, it, expect } from 'vitest';
+import { parseFilterAST } from '@objectstack/spec/data';
+
+import { match } from './memory-matcher.js';
+
+type Refusal = Error & { code?: string; status?: number };
+
+/**
+ * The card's own NUMERIC fixture, in both readings of "no value" — numeric
+ * because `null` coerces to `0` under a relational comparison, which is the
+ * coercion that split the two faces; a string fixture hides it (#13553).
+ */
+const NULLED_ROWS: Array<Record<string, unknown>> = [
+  { id: '1', n: 5 },
+  { id: '2', n: 0 },
+  { id: '3', n: null },
+];
+const MISSING_ROWS: Array<Record<string, unknown>> = [
+  { id: '1', n: 5 },
+  { id: '2', n: 0 },
+  { id: '4' },
+];
+
+/**
+ * The direct-caller pipeline: compile first, evaluate second. The refusal has
+ * to land in step one — if compile returns, the matcher HAS been reached and
+ * the pin below fails on the sentinel rather than on a missing throw.
+ */
+function compileThenMatch(rows: Array<Record<string, unknown>>, where: unknown): string[] {
+  const condition = parseFilterAST(where);
+  return rows.filter((row) => match(row, condition)).map((row) => String(row.id));
+}
+
+const refusalOf = (run: () => unknown): Refusal => {
+  try {
+    run();
+  } catch (e) {
+    return e as Refusal;
+  }
+  throw new Error('expected the compile face to refuse this filter, but it returned');
+};
+
+describe('[#14080] a refused null ordering comparand cannot reach the matcher (ruled 2026-09-01)', () => {
+  it.each([
+    ['$gt: null', { n: { $gt: null } }],
+    ['$gte: null', { n: { $gte: null } }],
+    ['$lt: null', { n: { $lt: null } }],
+    ['$lte: null', { n: { $lte: null } }],
+    ['lowered array form, ">="', [['n', '>=', null]]],
+    ['lowered array form, "before"', [['n', 'before', null]]],
+  ])('%s aborts at the compile face on BOTH readings of "no value"', (_label, where) => {
+    // Record-independent by construction — the compile face never sees a row —
+    // so the two readings that split the faces (the card's table) cannot even
+    // be posed. Driving both anyway is the point of the pin: neither fixture
+    // gets an answer, so there is no divergence left to observe.
+    for (const rows of [NULLED_ROWS, MISSING_ROWS]) {
+      const err = refusalOf(() => compileThenMatch(rows, where));
+      expect(err.code, _label).toBe('INVALID_FILTER');
+      expect(err.status, _label).toBe(400);
+    }
+  });
+
+  it('the pipeline itself is real — a legal ordering comparand compiles and the matcher answers', () => {
+    // Positive control: without it, the refusals above would also "pass" if
+    // compileThenMatch were broken outright. `0` is the discriminator the
+    // numeric fixture exists for — a VALUE, kept in, on every arm.
+    expect(compileThenMatch(NULLED_ROWS, { n: { $gt: 0 } })).toEqual(['1']);
+    expect(compileThenMatch(NULLED_ROWS, { n: { $gte: 0 } })).toEqual(['1', '2']);
+    expect(compileThenMatch(MISSING_ROWS, { n: { $lt: 5 } })).toEqual(['2']);
+    expect(compileThenMatch(MISSING_ROWS, [['n', '<=', 0]])).toEqual(['2']);
+  });
+
+  it('the null PREDICATE still passes the same face — the refusal is ordering-shaped, not null-shaped', () => {
+    // `$eq: null` IS the null predicate on both readings (#13494) and is the
+    // spelling the refusal prescribes; the carve-out must not catch it.
+    expect(compileThenMatch(NULLED_ROWS, { n: { $eq: null } })).toEqual(['3']);
+    expect(compileThenMatch(MISSING_ROWS, { n: { $eq: null } })).toEqual(['4']);
+    expect(compileThenMatch(NULLED_ROWS, { n: { $ne: null } })).toEqual(['1', '2']);
+  });
+});

packages/objectql/src/engine-filter-array-lowering.test.ts

@@ -596,6 +596,79 @@ describe('Door 2 lowers FilterArray to FilterCondition before the driver (#5158)
     expect(reads).toHaveLength(0);
   });
 
+  // ── [#14080] the ORDERING carve-out, ruled 2026-09-01: a null comparand ──
+  // ── of $gt/$gte/$lt/$lte is refused at this seam, so driver-memory's ─────
+  // ── two-face divergence on it is UNREACHABLE through the engine ─────────
+  //
+  // Ruling point 4's negative pin, engine half, in the exact shape of the
+  // #13357 block above: the witness is the recording driver's call log, not
+  // the thrown envelope alone. The compile-face half (`parseFilterAST`, both
+  // input forms) is pinned in `@objectstack/spec`'s
+  // `filter-comparand-shape.test.ts`; the matcher-side statement lives in
+  // driver-memory's `memory-null-ordering-comparand-unreachable.test.ts`.
+  // ⛔ Nothing here asserts what either face WOULD have answered, and no
+  // ordering-vs-null semantics is defined — the divergence is sealed.
+
+  it.each([
+    ['$gt: null', { amount: { $gt: null } }],
+    ['$gte: null', { amount: { $gte: null } }],
+    ['$lt: null', { amount: { $lt: null } }],
+    ['$lte: null', { amount: { $lte: null } }],
+    ['lowered array form, ">="', [['amount', '>=', null]]],
+  ])('a null ordering comparand is refused on EVERY verb before any driver call — %s', async (_l, where) => {
+    // `asFilterArrayQuery`: the array-form case makes `where` off-contract by
+    // declaration (see the helper's note), and the spelling names that.
+    await expect(engine.find('deal', asFilterArrayQuery(where)))
+      .rejects.toMatchObject({ status: 400, code: 'INVALID_FILTER' });
+    await expect(engine.findOne('deal', asFilterArrayQuery(where)))
+      .rejects.toMatchObject({ status: 400, code: 'INVALID_FILTER' });
+    await expect(engine.count('deal', { where } as unknown as EngineCountOptions))
+      .rejects.toMatchObject({ status: 400, code: 'INVALID_FILTER' });
+    await expect(engine.aggregate('deal', {
+      where: where as unknown as EngineAggregateOptions['where'],
+      groupBy: ['stage'],
+      aggregations: [{ function: 'count', field: 'id', alias: 'n' }],
+    })).rejects.toMatchObject({ status: 400, code: 'INVALID_FILTER' });
+    await expect(engine.update('deal', { amount: 1 }, { where, multi: true } as any))
+      .rejects.toMatchObject({ status: 400, code: 'INVALID_FILTER' });
+    await expect(engine.delete('deal', { where, multi: true } as any))
+      .rejects.toMatchObject({ status: 400, code: 'INVALID_FILTER' });
+    // The negative half: refused BEFORE the store — no read, no write, no row
+    // moved. (The count() control below adds its own read, so it runs after.)
+    expect(reads).toHaveLength(0);
+    expect(writes).toHaveLength(0);
+    expect(await engine.count('deal')).toBe(3);
+  });
+
+  it('the null-ordering refusal is not vacuous — the same operator WITHOUT null reaches the driver', async () => {
+    // Positive control for the zero-call reading above: one comparand
+    // swapped for a value, same operator, same field, and the dispatch happens.
+    const rows = await engine.find('deal', { where: { amount: { $gt: 10 } } });
+    expect(reads).toHaveLength(1);
+    expect(lastWhere()).toEqual({ amount: { $gt: 10 } });
+    expect(rows.map((r: any) => r.id).sort()).toEqual(['d2', 'd3']);
+  });
+
+  it('the null PREDICATE still reaches the driver — the refusal is ordering-shaped, not null-shaped', async () => {
+    // `$eq: null` / `$ne: null` ARE the null predicate (#5332) and are the
+    // spellings the refusal prescribes; the seam must keep passing them.
+    await engine.find('deal', { where: { owner_id: { $ne: null } } });
+    expect(reads).toHaveLength(1);
+    expect(lastWhere()).toEqual({ owner_id: { $ne: null } });
+  });
+
+  it('a nested null ordering comparand is refused at its own path, engine prefix and all', async () => {
+    const err = await engine.find(
+      'deal',
+      { where: { $or: [{ amount: { $lte: null } }] } },
+    ).then(() => null, (e: any) => e);
+    expect(err?.status).toBe(400);
+    expect(err?.code).toBe('INVALID_FILTER');
+    expect(err.message).toMatch(/^find\('deal'\): /);
+    expect(err.message).toContain('where.$or[0].amount.$lte');
+    expect(reads).toHaveLength(0);
+  });
+
   // ── what must KEEP working: the declared list shapes ───────────────────
 
   it('a proper list comparand still reaches the driver untouched', async () => {