Skip to content

feat(driver-sql,driver-turso,driver-sqlite-wasm): SQLite-family JSON columns declare TEXT; server dialects keep native JSON - #13442

Merged
zhuangjianguo merged 6 commits into
mainfrom
claude/issue-12738-sqlite-json-columns-declare-text
Aug 30, 2026
Merged

feat(driver-sql,driver-turso,driver-sqlite-wasm): SQLite-family JSON columns declare TEXT; server dialects keep native JSON#13442
zhuangjianguo merged 6 commits into
mainfrom
claude/issue-12738-sqlite-json-columns-declare-text

Conversation

@claude

@claude claude Bot commented Aug 30, 2026

Copy link
Copy Markdown
Contributor

Fixes #12738

Implements ruling C' (maintainer, 2026-08-28, recorded on the card): each dialect declares the semantically correct JSON column type. Neither of the card's own two directions is taken.

What changed

SqlDriver.createColumn had two JSON routes, both hard-wired to knex's table.json(name) on every dialect. They now go through one seam, SqlDriver.jsonColumn, which emits table.text(name) on the SQLite family and table.json(name) everywhere else.

dialect before after
Postgres json json — untouched
MySQL json json — untouched
SQLite family json text

Pre-work verification (ruling fence 1) — read, then measured

The C' scope claim rests on knex's actual DDL and on which dialects share the arm. Both were measured, not assumed.

knex's t.json(), read in source then compiled: the base compiler is already correct — ColumnCompiler.prototype.bit = ColumnCompiler.prototype.json = 'text'. The sqlite3 dialect is the one that overrides it: ColumnCompiler_SQLite3.prototype.json = 'json'. Compiled DDL for t.json('v'):

better-sqlite3   create table `probe` (`v_json` json, `v_text` text, `v_str` varchar(255))
sqlite3          create table `probe` (`v_json` json, `v_text` text, `v_str` varchar(255))
pg               create table "probe" ("v_json" json, "v_text" text, "v_str" varchar(255))
mysql2           create table `probe` (`v_json` json, `v_text` text, `v_str` varchar(255))

So json on SQLite names a type the engine does not have. SQLite derives affinity from substrings of the declared type name, and json contains none of the markers (INT, CHAR/CLOB/TEXT, BLOB, REAL/FLOA/DOUB), so it falls through to NUMERIC. Measured through raw SQL on such a column: a bare '0123' is stored as the integer 123.

Which dialects reach the arm. The two createColumn sites are shared by every dialect, so the discriminator is this.isSqlite. Its coverage was verified rather than presumed:

  • plain SqlDriver on sqlite3 / sqlite / better-sqlite3;
  • TursoDriver in all three transport modes — local, embedded-replica and remote all configure client: 'better-sqlite3';
  • SqliteWasmDriver, which passes a knex Client constructor (so the string match answers '') and therefore overrides isSqlite to true for exactly this reason. An inline config.client string test written at the call site would have silently missed the sibling SQLite dialect.

One of the three named sites is deliberately NOT changed

The dispatch named three JSON-mapping sites. I changed two. buildRebuiltColumn (sql-driver.ts:11096) maps an introspected existing column type back to a builder for a SQLite table rebuild — its input is a column's current declaration, not a field's metadata. Routing it through the new emitter would make an unrelated drift rebuild (a relaxed NOT NULL, a dropped column) re-declare a legacy json column as text and, because the rebuild's insert ... select copies rows through the new affinity, rewrite the stored bytes of live data. That contradicts the ruling's own migration shape, so the arm keeps its behaviour and now carries the reason in code. Columns created after this change arrive there declared text and take the text arm, so the rebuild is stable in both eras with no special-casing.

Instrument: affinity-level, not VALUE_ROUNDTRIP

The card's own negative result is that the shared round-trip case-set is green on both sides of this decision, so it cannot adjudicate direction. The pins added here assert, on both transports: the declared type from the catalog, typeof(col) storage class per case, and a raw-SQL bare '0123' that must survive as text.

Measured convergence, from the pin itself: the local/remote pair moved from ['json', 'TEXT'] to ['TEXT', 'TEXT'], and the list of cases whose storage class differs between the transports went from two entries to empty.

A detail worth recording: knex writes `v_json` text (lower case) into the stored CREATE TABLE, and pragma_table_info reports it back as TEXT — so the two transports turn out to declare the byte-identical type string, not merely the same affinity class. The pin still asserts affinity as the primary property, because that is what matters and an exact-string pin would go red over a cosmetic re-spelling.

Pins inverted in place, never re-baselined

Three suites pinned the old design. Each is inverted with the reason recorded, and each keeps a control that can still fail:

  • turso-json-column-type-asymmetry.test.ts — inverted per its own header, which instructed exactly this. It now pins the convergence. A new negative control creates a legacy json column by hand and shows NUMERIC affinity is still real on the same SQLite build, so the "survives as text" probe cannot pass vacuously. The filename is kept: two source comments and a landed changeset refer to it by name.
  • sql-driver-12380-json-roundtrip.test.ts — the catalog-type assertion is now dialect-aware. The affinity block is split: §A the column the driver emits today (text, no NUMERIC affinity), §B a legacy json column where the original [finding] driver-sql's SQLite json codec is not round-trip faithful for a string whose content is valid JSON — '123' reads back as a number, and Postgres does not agree #12380 measurements are asserted unchanged. That is the ruling's requirement that those pins stay green and stay in force for legacy columns. The legacy-corpus migration suite now builds its table from the driver's own CREATE TABLE with val put back to json, and asserts the rewrite landed — otherwise the corpus would have quietly contained no legacy cells at all and every assertion would have passed while measuring nothing.
  • schema-drift.base-type-mismatch.test.ts — the fresh-table type expectation is dialect-aware. The inversion reinforces the suite: SQLite was already excluded from the multi-value base-type finding because the column type is not load-bearing there, and the emitter now agrees with the differ instead of merely being excused by it.

Migration shape

New columns only. Existing columns keep their declared type, keep NUMERIC affinity, and keep the injective codec that defends them. Platform write-path behaviour is unchanged. No new schema-drift findings. Nothing on the read path consults the physical column type — isJsonField answers from metadata. The only visible difference is raw-SQL-only and in the safe direction: a value that would previously have been coerced to a number is now preserved as text; nothing preserved before is coerced now. Stated in the MINOR changeset.

Unmeasured premise, passed through undiluted

The ruling flagged its own: "The replica DDL-sync detail is reasoned from the product model, not measured; the dev verifies before relying on it." It remains unmeasured — exercising embedded-replica schema sync needs a live Turso primary, which this container has none of, and the libsql test stub is a local better-sqlite3 double with no sync. Nothing here relies on it. C' is justified entirely by the measured affinity trap. The change also makes the question moot in the direction it was raised: all three turso modes now declare TEXT, so a deployment's physical schema no longer depends on which side ran DDL.

Verification at 8058e2b606

pnpm --filter driver-sql --filter driver-turso --filter driver-sqlite-wasm test        exit 0
  driver-sql          2248 passed | 132 skipped   (147 files passed, 9 skipped)
  driver-turso        1108 passed                 (41 files passed)
  driver-sqlite-wasm   446 passed                 (26 files passed)
pnpm --filter (same three) typecheck                                                   exit 0
pnpm lint   (eslint . --no-inline-config, whole repo, not narrowed)                     exit 0

29 derived gate families green, re-derived against the actual diff with node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack after merging origin/main to clear a stale-tree warning. Includes check:driver-conformance, check:test-source-alias, check:engine-double-contract, check:where-matcher, check:cross-package-test-inputs, check:dispatcher-error-vocabulary, check:nul-bytes, and the check:type-check-debt ratchet, which was run only after building the full package closure — it refuses outright on an unbuilt worktree, and that refusal is NOT MEASURED rather than green. It reports 29 ledger entries re-measured, none above its recorded number.

The two typecheck scripts were confirmed with tsc --listFiles to actually compile the three edited test files, so "typecheck clean" covers the edits rather than skipping them.


Generated by Claude Code

claude added 6 commits August 30, 2026 11:07
Each dialect now declares the semantically correct JSON column type.
Postgres and MySQL keep their native JSON types; the SQLite family
(plain SqlDriver on sqlite3/better-sqlite3, TursoDriver in all three
transport modes, and SqliteWasmDriver) declares text.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F3jdziLbAPGeceVNmSox5L
@github-actions

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

This PR changes 1 package(s): @objectstack/driver-sql, touching 3 documentable anchor(s).

6 hand-written doc(s) NAME something this change touched and may need an implementation-accuracy re-verification:

  • content/docs/data-modeling/drivers.mdx (via SqlDriver (symbol))
  • content/docs/data-modeling/index.mdx (via SqlDriver (symbol))
  • content/docs/plugins/packages.mdx (via SqlDriver (symbol))
  • content/docs/protocol/kernel/index.mdx (via SqlDriver (symbol))
  • content/docs/protocol/kernel/lifecycle.mdx (via SqlDriver (symbol))
  • content/docs/protocol/objectql/query-syntax.mdx (via SqlDriver (symbol))

1 release-owned page(s) also name something this change touched. These are read-only:

  • content/docs/releases/v17.mdx (via SqlDriver (symbol))

content/docs/releases/ is RELEASE-OWNED (AGENTS.md "Documentation Guardrails"): release
notes are written centrally at release time, and a code PR that edits them is the exact PR
that guardrail exists to stop. They are still audited — read-only. If one of them is actually
wrong, file an issue or open a dedicated docs-only PR; do not edit it here.

What this run could not see
  • the SDK route bridge reached 47 of 219 client-bound route-ledger rows — the other 172 have no registrar path: tail to select them, so pages documenting THEIR client methods cannot appear above, on this or any run. Of those 172: 14 are remediable by widening that discovery convention (an in-repo file declares the path; the convention did not scan it); 56 are structural — on a ledger where NOT ONE row is declared in-repo, so no discovery change reaches them at any price; 102 are undecided (no in-repo declaration, on a ledger that has other in-repo registrars — absence and an unreadable spelling are not distinguishable here). The rows themselves: node scripts/docs-audit/affected-docs.mjs --bridge-coverage
  • a page that states a rule by its inputs shares no identifier with the emitter that implements the rule, so an emitter-only diff cannot list it — not on this run and not on any run. Measured on fix(driver-sql): emit varchar(maxLength) for a text field a declared index keys on #11430: content/docs/protocol/objectql/types.mdx documents the text-family column mapping by the ObjectQL type names it maps FROM (text / textarea / html) while the diff changed createColumn; it went unlisted, and it was the page that diff falsified, in four places. No shared token exists to detect this on, so a rule your change carries has to be re-read by hand in the pages that restate it.

Coarse fallback — 9 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json bac201e9288bafbf23b95638aa2b48761effdea6packageMentionDocs.

Which tree this was computed on

This run read content/docs from 706d2148b09c13ca4c6234f4db2b03e36ea1e313 — the merge of head 8058e2b606a94776d49be771f6f4a7ea491a498f into base bac201e9288bafbf23b95638aa2b48761effdea6, which is what actions/checkout gives a pull_request run. Not the PR head.

A worktree cut from an older main holds a different content/docs, so re-deriving there can legitimately return a different list — that is a different tree, not a wrong row. To answer on the same tree:

# while this PR is open — GitHub drops the merge commit once it closes
git fetch origin 706d2148b09c13ca4c6234f4db2b03e36ea1e313 && git checkout 706d2148b09c13ca4c6234f4db2b03e36ea1e313
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin bac201e9288bafbf23b95638aa2b48761effdea6 8058e2b606a94776d49be771f6f4a7ea491a498f && git checkout -B drift-repro bac201e9288bafbf23b95638aa2b48761effdea6 && git merge --no-ff 8058e2b606a94776d49be771f6f4a7ea491a498f

node scripts/docs-audit/affected-docs.mjs --json bac201e9288bafbf23b95638aa2b48761effdea6

⚠️ That checkout carried uncommitted changes, so the commit above does not fully identify what was read.

Advisory only, and a precision-first one (#9192): a page is listed because it names a
symbol, wire route or SDK method this diff touched — not because it mentions a changed
package. Each row says which anchor put it there, so a wrong row is reportable rather than
merely annoying. To re-verify, run the docs-accuracy-audit workflow scoped to these files:
node scripts/docs-audit/affected-docs.mjs bac201e9288bafbf23b95638aa2b48761effdea6 → pass the list as
args.docs, on the commit named under Which tree this was computed on.

@github-actions github-actions Bot added documentation Improvements or additions to documentation tests tooling labels Aug 30, 2026
@zhuangjianguo
zhuangjianguo marked this pull request as ready for review August 30, 2026 12:38
@zhuangjianguo
zhuangjianguo added this pull request to the merge queue Aug 30, 2026
Merged via the queue into main with commit 9f4a6d5 Aug 30, 2026
34 checks passed
@zhuangjianguo
zhuangjianguo deleted the claude/issue-12738-sqlite-json-columns-declare-text branch August 30, 2026 12:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation size/l tests tooling

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Converging driver-turso's two transports onto one physical column type for Field.json — and the measured reason VALUE_ROUNDTRIP cannot adjudicate it

2 participants