feat(driver-sql,driver-turso,driver-sqlite-wasm): SQLite-family JSON columns declare TEXT; server dialects keep native JSON - #13442
Conversation
…lite-json-columns-declare-text
Each dialect now declares the semantically correct JSON column type. Postgres and MySQL keep their native JSON types; the SQLite family (plain SqlDriver on sqlite3/better-sqlite3, TursoDriver in all three transport modes, and SqliteWasmDriver) declares text. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01F3jdziLbAPGeceVNmSox5L
📓 Docs Drift CheckThis PR changes 1 package(s): 6 hand-written doc(s) NAME something this change touched and may need an implementation-accuracy re-verification:
⛔ 1 release-owned page(s) also name something this change touched. These are read-only:
What this run could not see
Coarse fallback — 9 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): Which tree this was computed onThis run read A worktree cut from an older # while this PR is open — GitHub drops the merge commit once it closes
git fetch origin 706d2148b09c13ca4c6234f4db2b03e36ea1e313 && git checkout 706d2148b09c13ca4c6234f4db2b03e36ea1e313
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin bac201e9288bafbf23b95638aa2b48761effdea6 8058e2b606a94776d49be771f6f4a7ea491a498f && git checkout -B drift-repro bac201e9288bafbf23b95638aa2b48761effdea6 && git merge --no-ff 8058e2b606a94776d49be771f6f4a7ea491a498f
node scripts/docs-audit/affected-docs.mjs --json bac201e9288bafbf23b95638aa2b48761effdea6
|
Fixes #12738
Implements ruling C' (maintainer, 2026-08-28, recorded on the card): each dialect declares the semantically correct JSON column type. Neither of the card's own two directions is taken.
What changed
SqlDriver.createColumnhad two JSON routes, both hard-wired to knex'stable.json(name)on every dialect. They now go through one seam,SqlDriver.jsonColumn, which emitstable.text(name)on the SQLite family andtable.json(name)everywhere else.jsonjson— untouchedjsonjson— untouchedjsontextPre-work verification (ruling fence 1) — read, then measured
The C' scope claim rests on knex's actual DDL and on which dialects share the arm. Both were measured, not assumed.
knex's
t.json(), read in source then compiled: the base compiler is already correct —ColumnCompiler.prototype.bit = ColumnCompiler.prototype.json = 'text'. The sqlite3 dialect is the one that overrides it:ColumnCompiler_SQLite3.prototype.json = 'json'. Compiled DDL fort.json('v'):So
jsonon SQLite names a type the engine does not have. SQLite derives affinity from substrings of the declared type name, andjsoncontains none of the markers (INT,CHAR/CLOB/TEXT,BLOB,REAL/FLOA/DOUB), so it falls through to NUMERIC. Measured through raw SQL on such a column: a bare'0123'is stored as the integer123.Which dialects reach the arm. The two
createColumnsites are shared by every dialect, so the discriminator isthis.isSqlite. Its coverage was verified rather than presumed:SqlDriveronsqlite3/sqlite/better-sqlite3;TursoDriverin all three transport modes — local, embedded-replica and remote all configureclient: 'better-sqlite3';SqliteWasmDriver, which passes a knex Client constructor (so the string match answers'') and therefore overridesisSqlitetotruefor exactly this reason. An inlineconfig.clientstring test written at the call site would have silently missed the sibling SQLite dialect.One of the three named sites is deliberately NOT changed
The dispatch named three JSON-mapping sites. I changed two.
buildRebuiltColumn(sql-driver.ts:11096) maps an introspected existing column type back to a builder for a SQLite table rebuild — its input is a column's current declaration, not a field's metadata. Routing it through the new emitter would make an unrelated drift rebuild (a relaxed NOT NULL, a dropped column) re-declare a legacyjsoncolumn astextand, because the rebuild'sinsert ... selectcopies rows through the new affinity, rewrite the stored bytes of live data. That contradicts the ruling's own migration shape, so the arm keeps its behaviour and now carries the reason in code. Columns created after this change arrive there declaredtextand take thetextarm, so the rebuild is stable in both eras with no special-casing.Instrument: affinity-level, not
VALUE_ROUNDTRIPThe card's own negative result is that the shared round-trip case-set is green on both sides of this decision, so it cannot adjudicate direction. The pins added here assert, on both transports: the declared type from the catalog,
typeof(col)storage class per case, and a raw-SQL bare'0123'that must survive as text.Measured convergence, from the pin itself: the local/remote pair moved from
['json', 'TEXT']to['TEXT', 'TEXT'], and the list of cases whose storage class differs between the transports went from two entries to empty.A detail worth recording: knex writes
`v_json` text(lower case) into the storedCREATE TABLE, andpragma_table_inforeports it back asTEXT— so the two transports turn out to declare the byte-identical type string, not merely the same affinity class. The pin still asserts affinity as the primary property, because that is what matters and an exact-string pin would go red over a cosmetic re-spelling.Pins inverted in place, never re-baselined
Three suites pinned the old design. Each is inverted with the reason recorded, and each keeps a control that can still fail:
turso-json-column-type-asymmetry.test.ts— inverted per its own header, which instructed exactly this. It now pins the convergence. A new negative control creates a legacyjsoncolumn by hand and shows NUMERIC affinity is still real on the same SQLite build, so the "survives as text" probe cannot pass vacuously. The filename is kept: two source comments and a landed changeset refer to it by name.sql-driver-12380-json-roundtrip.test.ts— the catalog-type assertion is now dialect-aware. The affinity block is split: §A the column the driver emits today (text, no NUMERIC affinity), §B a legacyjsoncolumn where the original [finding] driver-sql's SQLite json codec is not round-trip faithful for a string whose content is valid JSON — '123' reads back as a number, and Postgres does not agree #12380 measurements are asserted unchanged. That is the ruling's requirement that those pins stay green and stay in force for legacy columns. The legacy-corpus migration suite now builds its table from the driver's ownCREATE TABLEwithvalput back tojson, and asserts the rewrite landed — otherwise the corpus would have quietly contained no legacy cells at all and every assertion would have passed while measuring nothing.schema-drift.base-type-mismatch.test.ts— the fresh-table type expectation is dialect-aware. The inversion reinforces the suite: SQLite was already excluded from the multi-value base-type finding because the column type is not load-bearing there, and the emitter now agrees with the differ instead of merely being excused by it.Migration shape
New columns only. Existing columns keep their declared type, keep NUMERIC affinity, and keep the injective codec that defends them. Platform write-path behaviour is unchanged. No new schema-drift findings. Nothing on the read path consults the physical column type —
isJsonFieldanswers from metadata. The only visible difference is raw-SQL-only and in the safe direction: a value that would previously have been coerced to a number is now preserved as text; nothing preserved before is coerced now. Stated in the MINOR changeset.Unmeasured premise, passed through undiluted
The ruling flagged its own: "The replica DDL-sync detail is reasoned from the product model, not measured; the dev verifies before relying on it." It remains unmeasured — exercising embedded-replica schema sync needs a live Turso primary, which this container has none of, and the libsql test stub is a local better-sqlite3 double with no sync. Nothing here relies on it. C' is justified entirely by the measured affinity trap. The change also makes the question moot in the direction it was raised: all three turso modes now declare
TEXT, so a deployment's physical schema no longer depends on which side ran DDL.Verification at
8058e2b60629 derived gate families green, re-derived against the actual diff with
node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstackafter mergingorigin/mainto clear a stale-tree warning. Includescheck:driver-conformance,check:test-source-alias,check:engine-double-contract,check:where-matcher,check:cross-package-test-inputs,check:dispatcher-error-vocabulary,check:nul-bytes, and thecheck:type-check-debtratchet, which was run only after building the full package closure — it refuses outright on an unbuilt worktree, and that refusal is NOT MEASURED rather than green. It reports 29 ledger entries re-measured, none above its recorded number.The two
typecheckscripts were confirmed withtsc --listFilesto actually compile the three edited test files, so "typecheck clean" covers the edits rather than skipping them.Generated by Claude Code