Skip to content

feat(spec,driver-sql,formula): addDays whole-day offset on a field reference, compiled on SQL and evaluated in memory - #15102

Merged
zhuangjianguo merged 4 commits into
mainfrom
claude/issue-14104-field-reference-add-days
Sep 3, 2026
Merged

feat(spec,driver-sql,formula): addDays whole-day offset on a field reference, compiled on SQL and evaluated in memory#15102
zhuangjianguo merged 4 commits into
mainfrom
claude/issue-14104-field-reference-add-days

Conversation

@zhuangjianguo

Copy link
Copy Markdown
Collaborator

Fixes #14104

Clause ②: yes — path limb: packages/spec/src/data/filter.zod.ts (a published accept set widens — FieldReferenceSchema gains addDays, regenerated into the JSON schema, the authorable surface and content/docs/references/data/filter.mdx); content limb: the filter grammar gains a key an author can write, so os validate accepts a shape it refused yesterday and two published runtimes (driver-sql, formula) give it a meaning.

What this lands (maintainer ruling A, 2026-09-02, recorded on #14104)

{ completed_at: { $lte: { $field: 'due_date', addDays: { $field: 'grace_days' } } } } — a whole-day offset on a field reference, so a dataset measure can say "completed by its deadline, where the deadline is a stored date plus a grace period held in another column".

  • SpecFieldReferenceSchema.addDays: an integer literal of any sign (a negative value subtracts; no subDays, no other unit) or a nested { $field } reference (dot-path allowed, exactly as $field allows). No new position: the reference is legal exactly where a bare one is (the whole comparand of the six scalar comparison operators); $in / $nin / $between keep the list-position refusal. A fractional number, a string, or an object without $field is refused at the schema door with a pointed first sentence, and the ordering slot repeats that sentence instead of zod's generic union text. The "Execution support" docblock is rewritten to the landed state (SQL push-down has compiled $field since PR feat(driver-sql): compile $field to column-to-column comparison (#5222) #7582; the offset rides the same arm). No transform, so no ADR-0122 XParsed alias.
  • SQL (driver-sql, inherited unchanged by driver-sqlite-wasm) — the PR feat(driver-sql): compile $field to column-to-column comparison (#5222) #7582 cross-field arm reads the offset off the same comparand node and compiles column OP (referenced + offset days) per dialect: SQLite date(col, 'N days') for a date column and strftime('%Y-%m-%dT%H:%M:%fZ', col, 'N days') for a datetime column (the driver's canonical text form, so a shifted value is byte-identical to a stored one and the compare stays a plain text compare — no julian-day arithmetic anywhere, so no stop); PostgreSQL date + integer / timestamptz + make_interval(days => n); MySQL date_add(col, interval n day). The offset value is coalesce(offset, 0), truncated toward zero on every dialect; a literal binds as a parameter where the column would. The offset rides the four [spec] SqlDriver 将 $field 编译为列对列比较(cross-field comparison push-down) #5222 rulings (same-table, declared-only, tenant column forbidden, same class) and adds two: day arithmetic compiles only between two date columns or two datetime columns, and the offset column must be numeric. Everything else is INVALID_FILTER / 400, operands withheld from the caller and named in the server log.
  • Memory (formula matchesFilter) — resolveValue shifts the referenced value by the resolved whole days in the shape it arrived in (a calendar day stays a calendar day, so $lte still covers the whole shifted day; an ISO instant keeps its time of day; Date and epoch number likewise); the offset is truncated toward zero, matching the dialects.
  • NULL semantics, stated in the describe and pinned on both paths in the shape $not 的语义在 driver-sql 与 driver-memory / formula 之间分叉:NULL 行的去留相反,$not: {} 一个是 TRUE 一个是 FALSE #5146 used for $not:
Case Both paths answer
offset column NULL zero days — due_date + NULL is due_date
referenced column NULL the comparison is FALSE for every operator, $ne included — no deadline is never "on time"; $not re-admits the row (the predicate is total)
target column NULL its ordinary reading — fails the orderings and $eq, satisfies $ne when the offset deadline exists

In SQL that is (referenced IS NOT NULL AND target IS NOT NULL AND target OP shifted) for the orderings and $eq, and (referenced IS NOT NULL AND (target IS NULL OR target != shifted)) for $ne; in memory a NULL base resolves to a sentinel evalOp answers false for before any operator runs. So with an offset the $eq/$ne pair is deliberately NOT the both-NULL-matching pair the bare arm emits — a row with no due date does not "equal" its deadline.

  • $not composition, measured and pinned — the cross-field arm was already classified null-total (operatorIsNullTotal), so the $not 的语义在 driver-sql 与 driver-memory / formula 之间分叉:NULL 行的去留相反,$not: {} 一个是 TRUE 一个是 FALSE #5146 negation rewrite leaves it unguarded and NOT is the exact complement. The corpus pins $not of $lte / $eq / $ne with a column offset on both paths; rows with no deadline are in every $not set and in no positive set.
  • Conformance corpus (cross-field-conformance-cases.ts) — a second, nine-row fixture (cross_field_task: a date pair and a datetime pair on the same calendar days at 12:00Z, a nullable numeric grace_days) carrying every NULL arrangement of target / base / offset, a negative grace, and a month boundary; 21 expectations generated for both pairs (42 cases) + 10 refusals. driver-sql runs it per DIALECT_CELLS (SQLite here; the live PG + MySQL cells are named skips locally and run in the Temporal Conformance (live PG + MySQL) job, which runs the whole driver-sql suite with both URLs); driver-sqlite-wasm runs it through its own sql.js dialect; the analytics face runs it through service.query (where and read scope).
  • Analytics routing — read and measured, pass-through, no fix needed. comparand-shape.ts isFieldReference is typeof value.$field === 'string' with extra keys ignored (mirroring driver-sql's fieldReferenceOf), so a reference carrying addDays is still detected and NativeSQLStrategy.canHandle still declines it to the engine path; pinned in cross-field-offset-dataset.test.ts together with findCrossFieldComparand under every combinator. The temporal comparand doors treat the object as never judged — measured at this head: packages/objectql/src/temporal-comparand-door.ts findUninterpretableTemporalComparand answers null for the column-offset, literal-offset and bare reference shapes on a datetime field and fires on the junk-string control; packages/core/src/utils/temporal-comparand.ts isUninterpretableTemporalComparand('datetime', ref) answers false for both shapes and true for the control. Neither package is edited. read-scope-sql.ts untouched.
  • Dataset pinpackages/services/service-analytics/src/__tests__/cross-field-offset-dataset.test.ts: a DatasetSchema dataset over the offset fixture with done_on_time (count, filter completed_on <= due_on + grace_days), late, the datetime-pair twin and a derived on_time_rate, driven through queryDataset against a real SqliteWasmDriver: 2 on time, 4 late, rate 2/9, the datetime twin the same 2, grouped by a dimension without losing the filter; and, with both capabilities declared, only the offset-filtered passes decline native SQL (exactly one raw statement for the plain count). Home: beside the analytics service's own cross-field-engine-fallback.test.ts, not the packages/rest/src/analytics-dataset-* family — that family pins the caller's view of a refusal envelope through the route's catch with a failing driver double; this card's claim is about ROWS on a real engine, and the harness for that lives here.
  • The dotted spelling, stated honestly. The ruling's literal shape names duty.grace_days, a relation path. The memory evaluator walks it (pinned in matches-filter-field-reference-offset.test.ts); SQL push-down refuses a dotted reference under the maintainer's 2026-08-06 same-table ruling on [spec] SqlDriver 将 $field 编译为列对列比较(cross-field comparison push-down) #5222 (no JOIN planning, no alias contract), and the offset inherits that rule rather than reopening it — INVALID_FILTER naming dotted path in the server log, pinned in the corpus refusals and at dataset level. So on a SQL deployment the dotted offset is a loud refusal, never a wrong number, and the same-table spelling is the one that answers on both paths. Recorded as an open question in the report for the PM.
  • Docs — hand-written content/docs/protocol/objectql/query-syntax.mdx gains a "Comparing Two Fields" section (the page had no $field paragraph to sit beside; the section documents the bare reference and the offset, with an os:check typed block that type-checks under check:skill-examples); generated content/docs/references/data/filter.mdx, the JSON schema, the authorable surface and the strictness-ledger counts via check:generated --fix.
  • Changesets@objectstack/spec minor, @objectstack/driver-sql minor, @objectstack/formula minor. driver-sqlite-wasm and service-analytics carry no changeset: only their tests moved.

Declared trigger-file touch (#6009)

packages/drivers/driver-sql/src/sql-driver.ts is on #6009's Restart-when (any PR touching sqliteCanonicalDatetimeSql or backfillCanonicalDatetimes). This diff does not change either function or any call to them. It does add a TSDoc {@link sqliteCanonicalDatetimeSql} mention in the new crossFieldOffsetExpr docblock (the SQLite day-add is applied on top of the canonical-text expression filterColumnExpr already returns for a not-yet-backfilled column), so a textual scan of the diff will hit the symbol name. Declared here for the hold's owner; no #6009 work is done in this PR.

Verification record — at b45c078 (git rev-parse --short HEAD after the last commit)

Every command run in the worktree at that head, exit captured before any pipe; verdict lines quoted from the tools' own output. Build closure first (pnpm --filter '@objectstack/driver-sql^...' build, pnpm --filter '@objectstack/service-analytics^...' build, then driver-sql, driver-sqlite-wasm, formula, spec, lint, client, client-react), heavy runs under scripts/pm/os-verify-lock.sh.

Check Result
pnpm --filter @objectstack/spec test Test Files 465 passed (465) · Tests 12408 passed (12408) — exit 0
pnpm --filter @objectstack/driver-sql test Test Files 152 passed, 9 skipped (161) · Tests 2345 passed, 140 skipped (2485) — exit 0; the skips are the unprovisioned live PG / MySQL cells, named by declareUnprovisionedCell (NOT MEASURED locally; the Temporal Conformance (live PG + MySQL) job runs them)
pnpm --filter @objectstack/driver-sqlite-wasm test Test Files 26 passed (26) · Tests 501 passed (501) — exit 0
pnpm --filter @objectstack/formula test Test Files 28 passed (28) · Tests 792 passed (792) — exit 0
pnpm --filter @objectstack/service-analytics test Test Files 89 passed (89) · Tests 1968 passed (1968) — exit 0
typecheck × spec, driver-sql, formula, driver-sqlite-wasm, service-analytics exit 0 each; spec's check:test-typecheck: OK — @objectstack/spec's test layer compiles (the new pins are inside its program)
pnpm --filter @objectstack/spec check:generated (second run, no --fix) ✓ All 15 generated artifacts are up to date.
check:authorable-surface / check:liveness / check:docs / check:api-surface ✅ Successfully generated 1600 schemas. · exit 0 · ✅ 229 generated files in sync · public API surface + factory signatures unchanged ✓
node scripts/check-system-context-census.mjs --fix then plain --fix: 0 anchor(s) rewritten · OK — 106 elevation read sites in 20 packages across 45 files, all anchored
eslint --no-inline-config over the 10 edited TypeScript files exit 0, no output
check:doc-authoring ✓ doc authoring guard: 14658 customer-facing string(s) across 723 spec sources clean (a first run at an earlier head flagged tracker ids in three new strings; stripped, with a negative pin added)
node scripts/pm/dispatch-gates.mjs --commands union — 79 commands (24 node, 55 pnpm) derived from the 17-path change set vs merge base 1c7adc7 74 green by their own verdict lines. NOT MEASURED by their own text: check-dev-prereqs.mjs (exit 1 — stale/absent dist/ of 58 unrelated packages, asks for a full pnpm build), check-test-completeness.mjs (exit 3 PREREQUISITE NOT MET — grades a saved turbo test log), check:dual-build-cjs-loads (exit 3 PREREQUISITE NOT MET — 76 packages without dist/), check:type-check-debt (exit 3 PREREQUISITE NOT MET — 25 ledgered deps unbuilt), check:entry-nameability (exit 0 but prints NOT MEASURED: no callable export on @objectstack/spec/qa, pre-existing). CI runs the farm.

Also run: check:skill-examples (✅ 257 prose examples type-check across 3 surface(s) — covers the new os:check block), check:strictness-ledger (is current — 439 site(s) measured), check:nul-bytes OK, check:cross-package-test-inputs OK, check:driver-conformance exit 0.

Reverse verification (two legs, restore proven by blob hash, script with trap restore)

  • Leg 1 — unhook the SQL offset compile (fieldReferenceOffsetOf forced to null; marker count on disk 1): sql-driver-cross-field-conformance.test.ts → 43 failed / 74 passed / 2 skipped; every red is in the [addDays] blocks — 17 distinct SQL push-down disagreed conformance cases plus the offset refusal arm (expected a refusal — the offset silently compiled as a bare comparison); 0 in-memory evaluator disagreed. The formula pins stayed green (28 passed) — memory untouched. Restored with git checkout HEAD --; git hash-object = HEAD blob 61008e7c…, marker count 0.
  • Leg 2 — unhook the memory offset (resolveValue returns the bare base; an executable string marker): matches-filter-field-reference-offset.test.ts → 20 failed / 8 passed (the 8 are the offset-free controls). Rebuilt formula; node scripts/ablation-dist-preflight.mjs @objectstack/formula 'ABLATION-14104-MEM'✓ dist/: marker present in 2 built files; the dist-resolving driver-sql conformance suite → 34 failed / 83 passed, every assertion in-memory evaluator disagreed, 0 SQL disagreements. Restored (hash-object = HEAD blob 087ba9fc…, marker count 0), rebuilt, preflight --absent✓ dist/: marker absent from all 6 built files, suite back to 117 passed / 2 skipped. (A first attempt used a comment marker the bundler strips — the preflight correctly refused it as "only in sourcemaps"; the leg was re-run with the executable marker above.)

🤖 Generated with Claude Code

https://claude.ai/code/session_019vx3536MUFc8XYVLNoKhs3


Generated by Claude Code

…corpus rows on both paths (wip)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019vx3536MUFc8XYVLNoKhs3
…tax section, changesets, regenerated spec artifacts (wip)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019vx3536MUFc8XYVLNoKhs3
…ference, compiled on SQL and evaluated in memory

FieldReferenceSchema gains addDays — an integer literal of any sign or a nested
{ $field } reference to a numeric column — so a dataset measure can express
completed_at <= due_date + grace_days. driver-sql compiles the offset on the
cross-field arm per dialect with the ruled NULL semantics written into the
predicate; matchesFilter evaluates it identically; the shared conformance corpus
carries the literal, column, negative, NULL-offset, NULL-base and $not rows on
both paths; the analytics detector keeps routing the reference to the engine
path and a dataset-level pin drives the on-time count on both.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019vx3536MUFc8XYVLNoKhs3
@github-actions

github-actions Bot commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

This PR changes 3 package(s): @objectstack/driver-sql, @objectstack/formula, @objectstack/spec, touching 55 documentable anchor(s). ⚠️ 2 changed file(s) yielded no anchor (packages/drivers/driver-sql/src/index.ts, packages/spec/authorable-surface/data.json), so the pages documenting them are NOT COVERED by this run — this is not a clean bill of health for those files.

48 hand-written doc(s) name something this change touched — list omitted above 15 rows. Re-derive on the tree named below: node scripts/docs-audit/affected-docs.mjs --json 29db3cd2ada4d88f97c52c00634609df9c573444.

4 release-owned page(s) also affected — read-only, see AGENTS.md Documentation Guardrails.

What this run could not see
  • 2 changed file(s) yielded no anchor (packages/drivers/driver-sql/src/index.ts, packages/spec/authorable-surface/data.json) — pages documenting those are invisible to this run
  • 9 name(s) were too generic to anchor anything (single lowercase words)
  • the SDK route bridge reached 47 of 219 client-bound route-ledger rows — the other 172 have no registrar path: tail to select them, so pages documenting THEIR client methods cannot appear above, on this or any run. Of those 172: 14 are remediable by widening that discovery convention (an in-repo file declares the path; the convention did not scan it); 56 are structural — on a ledger where NOT ONE row is declared in-repo, so no discovery change reaches them at any price; 102 are undecided (no in-repo declaration, on a ledger that has other in-repo registrars — absence and an unreadable spelling are not distinguishable here). The rows themselves: node scripts/docs-audit/affected-docs.mjs --bridge-coverage
  • a page that states a rule by its inputs shares no identifier with the emitter that implements the rule, so an emitter-only diff cannot list it — not on this run and not on any run. Measured on fix(driver-sql): emit varchar(maxLength) for a text field a declared index keys on #11430: content/docs/protocol/objectql/types.mdx documents the text-family column mapping by the ObjectQL type names it maps FROM (text / textarea / html) while the diff changed createColumn; it went unlisted, and it was the page that diff falsified, in four places. No shared token exists to detect this on, so a rule your change carries has to be re-read by hand in the pages that restate it.

Coarse fallback — 132 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json 29db3cd2ada4d88f97c52c00634609df9c573444packageMentionDocs.

Which tree this was computed on

This run read content/docs from cb9d4b8f06101bdb4c8a77c90974991fbd9d25bb — the merge of head b45c07858b2a27cdc0ab78b80f7aa1544e8cd2de into base 29db3cd2ada4d88f97c52c00634609df9c573444, which is what actions/checkout gives a pull_request run. Not the PR head.

A worktree cut from an older main holds a different content/docs, so re-deriving there can legitimately return a different list — that is a different tree, not a wrong row. To answer on the same tree:

# while this PR is open — GitHub drops the merge commit once it closes
git fetch origin cb9d4b8f06101bdb4c8a77c90974991fbd9d25bb && git checkout cb9d4b8f06101bdb4c8a77c90974991fbd9d25bb
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin 29db3cd2ada4d88f97c52c00634609df9c573444 b45c07858b2a27cdc0ab78b80f7aa1544e8cd2de && git checkout -B drift-repro 29db3cd2ada4d88f97c52c00634609df9c573444 && git merge --no-ff b45c07858b2a27cdc0ab78b80f7aa1544e8cd2de

node scripts/docs-audit/affected-docs.mjs --json 29db3cd2ada4d88f97c52c00634609df9c573444

⚠️ That checkout carried uncommitted changes, so the commit above does not fully identify what was read.

Advisory only, and a precision-first one (#9192): a page is listed because it names a
symbol, wire route or SDK method this diff touched — not because it mentions a changed
package. Each row says which anchor put it there, so a wrong row is reportable rather than
merely annoying. To re-verify, run the docs-accuracy-audit workflow scoped to these files:
node scripts/docs-audit/affected-docs.mjs 29db3cd2ada4d88f97c52c00634609df9c573444 → pass the list as
args.docs, on the commit named under Which tree this was computed on.

Copy link
Copy Markdown
Collaborator Author

Landing provenance (PM seat domain:spec, session_0174WZTU6XcFcS7g2kykC53i, 2026-09-03T22:57Z) — flipping to ready and enabling auto-merge (squash).


Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation protocol:data size/xl tests tooling

Projects

None yet

2 participants