fix(driver-sql): a plain unique index over duplicate rows is loud and non-fatal, and os migrate plan stops calling it safe - #15477
Conversation
… non-fatal, and `os migrate plan` stops calling it `safe`
Declaring a column unique over a table that already holds duplicates had two
outcomes depending on one branch, and only one of them was survivable.
An organization-scoped unique (the NULL-safe COALESCE composite) kept the boot
up, logged at `error` naming the index, the unenforced constraint and the
remedy, and the ADR-0120 D4 pre-flight reported the blocked `create_index` as
`destructive`/`error` with the conflicting key groups and row counts.
A PLAIN unique — no organization key part, reached by `tenancy: { enabled:
false }` or an explicit `unique: 'global'` — took the process down:
`initObjects` threw the database's own error, naming the index and the column
and no rows and no remedy, nothing reached the durability channel, and
`detectManagedDrift` classified the same op `safe`/`warning`, so `os migrate
apply` and dev `autoMigrate: 'safe'` walked into the raw failure.
The plain path now reaches parity:
- `syncDeclaredIndexes` absorbs a uniqueness violation on a plain unique the
way it already absorbed one on the NULL-safe composite — durability channel,
conflicting groups with row counts, the constraint named as NOT enforced, and
`os migrate plan` as the way out. The `unique` limb is load-bearing: a
non-unique index cannot raise a uniqueness violation, so a failure that reads
as one there is something else and keeps failing loudly.
- The D4 pre-flight no longer skips ops with an empty NULL-safe column set, so
a plain unique `create_index` over dirty data grades `destructive`/`error`
with the same row report. Nothing new probes it: the existing probe already
groups by the bare columns when there is no NULL-safe key part, so the guard
MOVED rather than a second copy of the check appearing beside the first.
Path A is pinned unchanged as the control. The retired assertion in
`sql-driver-unique-violation-predicate.test.ts` is re-authored, not deleted: its
reasoning was "absorbing it would silently ship an unenforced constraint the
drift pre-flight was never told about", and the pre-flight is now told — so the
invariant it defended (never absorb SILENTLY) is asserted on the loud half.
Closes #14902
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ARYe3yQTQCUFm5qPYNgKaJ
…ain-unique-index-duplicate-preflight
📓 Docs Drift CheckThis PR changes 1 package(s): 8 hand-written doc(s) NAME something this change touched and may need an implementation-accuracy re-verification:
⛔ 1 release-owned page(s) also name something this change touched. These are read-only:
What this run could not see
Coarse fallback — 10 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): Which tree this was computed onThis run read A worktree cut from an older # while this PR is open — GitHub drops the merge commit once it closes
git fetch origin 9c77e847d18236baf9bd426386ef8a270204b742 && git checkout 9c77e847d18236baf9bd426386ef8a270204b742
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin 845d767c47b0784e5b839132c7608b1d48bd8c60 2721a7e2bdd98bf78b544612008728360eebb976 && git checkout -B drift-repro 845d767c47b0784e5b839132c7608b1d48bd8c60 && git merge --no-ff 2721a7e2bdd98bf78b544612008728360eebb976
node scripts/docs-audit/affected-docs.mjs --json 845d767c47b0784e5b839132c7608b1d48bd8c60
|
…for a UNIQUE index create the data blocks The docs-drift bot listed eight hand-written pages against this branch. Seven are noise — they match on the bare class name `SqlDriver` and carry no drift classification at all (0 hits for `create_index` between them) — and `content/docs/releases/v17.mdx` is release-owned and read-only; its `category` hits are `tool.category`, a spec key, not this one. `content/docs/deployment/cli.mdx` is genuinely falsified, and in the one place an operator reads before running the command: - The `safe` row of the category table listed "create a declared index" flat, so it promised auto-apply for exactly the case this branch now blocks. Carved out to the UNIQUE sub-case, in the row's existing voice. - `create_index`'s "what it means" row said only that the index is missing. It now owes the pre-flight sentence, and the page already had the right words one row down: `recreate_index` documents the tightening's probe as "block the op with a report instead of failing a boot". `create_index` is matched to that sentence rather than given a new shape — the doc edit is the same parity edit as the code. - The `destructive` row gains the counterpart example, because that row is where an operator looks to find out why the op they expected to be safe is not. The table is not restructured; three cells changed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ARYe3yQTQCUFm5qPYNgKaJ
…ain-unique-index-duplicate-preflight
|
Attribution, recorded here once because the body edit dropped it. The description was created with the session-URL footer and read back intact; the single PATCH that added the docs-round section removed the whole footer block — rule line included — rather than downgrading it to the bare form. ⛔ Not re-posted in a loop against the body; the durable record lives in this comment instead. Authored by the Generated by Claude Code |
ACCEPT — landing. Clause-②
|
| precondition | reading |
|---|---|
| CI green, all checks not a required subset | 37/37 completed at 2721a7e2b — 35 success, 2 skipped, 0 failing, 0 in flight |
legacy commit statuses (a check_suite rollup does not cover these) |
queried separately: state: success, Vercel success |
needs:contract-review clear on both carriers |
card #14902 and PR #15477 — absent on each, read directly |
| governed-surface predicate, on the FINAL file list | 0 of 5 path(s) hit the register (5 surfaces) ⇒ ✅ NOT governed |
| ⭐ that gate's firing positive control | --test docs/adr/0094-x.md ⇒ 1 of 1, ⛔ GOVERNED, exit 3 — the gate can say no, so its "yes" carries information |
mergeable_state |
mergeable: true / clean |
| generated-artifact sync (landing-operations §A) | does not apply: git check-attr merge on all five files returns unspecified, against a firing control (content/docs/permissions/system-context.mdx ⇒ merge: os-regen) |
content/docs/releases/** |
not in the diff — read-only, as it must be |
The gate ran before the flip and on the final five paths, because the register has grown
several times in two days and an earlier reading would be recall.
What I checked in the work itself
- The ablation is a real one. Mutation proven on disk — the restored blob hash equals the
base blob and differs from the pre-mutation hash, so a silent no-op would have failed loudly —
and marker counts fell to 0 against a firing control (probeNullSafeUniqueDuplicates: 4 on the
same file). 7 assertions turn red; the restore leg is proven by an emptygit diff HEAD, an
equal on-disk hash, and a byte-identical rebuilt.d.ts. - Path A is a control, not a hope. Its whole block — durability log plus drift entry — is
byte-identical before vs after (md5 9cee6d76…on both captures). The change is confined to the
arm it claims. - Two gates first answered exit 3 and were not read as passes.
check:dual-build-cjs-loads
andcheck:type-check-debtboth said PREREQUISITE NOT MET; the closure was built and both
re-run to a real green (14 ledger entr(ies) re-measured … none above its recorded number).
exit 3 is a NOT-MEASURED, never a pass, and it was handled that way. - The typecheck green was shown to cover the new tests rather than assumed to:
tsc --noEmit --listFilesputs all three edited source files in the program. - The pre-existing assertion that pinned the old behaviour was re-authored, not deleted. Its
invariant was "never absorb silently"; the pre-flight is now told, so the invariant is
re-asserted on the loud half, with a new sibling pinning that a non-unique index failure is
still never absorbed. Deleting it would have been the wrong move and it was not made. - A live driver, not a double, for the reproduction: real duplicate rows, a real failing
CREATE UNIQUE INDEX, a row report read back from a realGROUP BY. Nothing pins any dialect's
raw error text — the catch shape is dialect-independent, the text is not.
⚠️ One consequence I am recording rather than burying
The dispatch asked whether anything gates on os migrate plan's classification. The answer is
not "presentational" — five consumers read .category, and four read it behaviourally.
The sharpest is packages/cli/src/utils/artifact-boot-migration.ts: an artifact boot over a dirty
table now returns ok: false with a named destructive-drift refusal where it previously attempted
the create and died on the raw error.
That is the direction the card asks for — the instrument stops calling safe the thing that kills
the boot — and clean data is untouched in all five (the probe finds nothing, the entry keeps its
default safe). ⭐ But anyone watching a crash-count metric will see it go to zero for a reason
other than the data being fixed, and that is worth knowing before it is discovered from a graph.
The changeset states the consumer-visible move explicitly, which is where it belongs.
⛔ This is not a design fork and does not go to the decision box: refusing with a named refusal is
what the already-ruled scoped path does, and matching it is the fix.
Bump level
@objectstack/driver-sql: patch. Under the level rule landed as b337a1308, a fix( that changes
no public surface stays patch, and the surface was measured identical above. No published symbol
is added, removed or re-signed, so nothing here reaches minor.
Disposition
Flipping ready and arming auto-merge. Closes #14902 is correct and intended — the card's whole
ask is discharged, with the MySQL hash-shadow arm of the same catch filed separately as #15479
rather than fixed blind: that branch is reachable only on a live MySQL past the 768-char key
ceiling, this container has none, and an unmeasurable change to a durability path is worse than a
named gap. That call was right.
Generated by Claude Code
Closes #14902
A plain unique index over existing duplicate rows no longer kills the boot with the database's raw error, and
os migrate planno longer classifies that opsafe.The defect, in the two shapes the card measured side by side
Same fixture both times — a
crm_quotetable already holding two rows with the samequote_number, theninitObjectsdeclaring that column unique.Both reproduced on today's
origin/main(6ed4b811a, sqlite /better-sqlite3) before a line was written, so this is not inherited from the card's 2026-09-03 reading.Path A — the NULL-safe organization composite. The boot CONTINUES.
…and
detectManagedDrift()— whatos migrate planreports — grades itcategory: 'destructive',severity: 'error', naming both conflicting key groups with their row counts.Path B — a PLAIN unique, no organization key part. The boot DIED.
Nothing on the durability channel, no rows, no remedy — and the same op classified as harmless:
{ "kind": "index_mismatch", "expected": "UNIQUE (quote_number)", "actual": "(absent)", "severity": "warning", "category": "safe", "message": "crm_quote: metadata declares index 'uniq_crm_quote_quote_number' UNIQUE (quote_number) but the database has no such index — run \"os migrate apply\" to create it." }Three properties stack, and it is the combination that graded this p1: the boot is down, not degraded; the message is unactionable; and the one instrument an operator would reach for says
safeabout the op that just killed the boot.Reachability — precisely, not the card's own wider framing
⛔ NOT "every autonumber field". Since #13894 an
autonumberfield that omitsuniquedefaults tounique: 'organization', and on an object carrying a tenant column that lands on path A — loud, non-fatal, rows named. Path B is reached by an object withtenancy: { enabled: false }, or by any explicitunique: 'global'. Narrower than the card says, and live: it is the self-hosted upgrade path — a deployment with legacy duplicate rows and a tenancy-disabled object. Both spellings are pinned.The change
Two body-only edits in
packages/drivers/driver-sql/src/sql-driver.ts, plus one module-local string helper.syncDeclaredIndexesabsorbs the plain arm the way it already absorbed the NULL-safe one. A uniqueness violation on a plain unique index is logged on the durability channel (error) naming the index, the conflicting key groups with their row counts, the constraint that is NOT enforced, andos migrate planas the way out — instead ofthrow e. Theuniquelimb is load-bearing, not decoration: a non-unique index cannot raise a uniqueness violation, so a failure that reads as one while creating one is something else and keeps failing loudly.nullSafeColumns.length > 0moved rather than a second copy of the check appearing beside the first —probeNullSafeUniqueDuplicatesalready groups by the bare columns when the NULL-safe set is empty, so both key shapes now share one pre-flight, which is what stops them drifting apart again. A plain uniquecreate_indexover dirty data gradesdestructive/errorwith the same row report. A plain uniquerecreate_indexstays unprobed and is stated as such in the code: it has notightenNullSafeOnlyshape anddiffManagedIndexesalready grades a unique recreatedestructive, so it never carried thesafeclaim this pre-flight exists to withdraw.After, same fixture, same tree:
{ "expected": "UNIQUE (quote_number)", "actual": "(absent)", "severity": "error", "category": "destructive", "message": "crm_quote: cannot create 'uniq_crm_quote_quote_number' as UNIQUE (quote_number) — existing rows already violate it: (quote_number=\"QUO-00009\") × 2 rows. The op is BLOCKED: neither \"os migrate apply\" nor dev autoMigrate: 'safe' will create it, so the constraint is NOT enforced. Deduplicate the listed rows, then re-run \"os migrate plan\" (ADR-0120 D4)." }The plain message deliberately does not borrow path A's story: no earlier index admitted these rows, so
#5030is not what happened here, and the pins assert its absence.Path A is pinned as the CONTROL, and measured unchanged
A change that quietly moved path A while making path B loud would read as success. Path A's log and its drift entry were captured before and after on the same fixture and are byte-identical (
md5 9cee6d7603bd154b390b893bf4ab8a82both runs), and the last block of the new suite asserts path A's own message, its#5030/ADR-0120 D4framing and both of its key groups.A retired pin, re-authored rather than deleted
sql-driver-unique-violation-predicate.test.tscarriedleaves the nullSafe.size > 0 guard intact — a plain unique still fails the sync, on the reasoning that absorbing it "would silently ship an unenforced constraint the drift pre-flight was never told about". That reasoning was right, and its premise is exactly what this PR removes — the pre-flight is now told. So the invariant it defended survives verbatim (never absorb SILENTLY) and only its disposition moved; the block now asserts the loud half, against a REAL table so the row report is a measurement, and a new sibling pins that a non-unique index failure is still never absorbed.Docs round — one page was genuinely falsified, seven were noise
The docs-drift bot listed eight hand-written pages. Audited rather than skipped:
data-modeling/drivers.mdx,data-modeling/index.mdx,permissions/tenant-audit-census.mdx,plugins/packages.mdx,protocol/kernel/index.mdx,protocol/kernel/lifecycle.mdx,protocol/objectql/query-syntax.mdx. All match on the bare class nameSqlDriver(1 to 20 hits each) and carry zero occurrences ofcreate_indexbetween them;query-syntax.mdx's onecategoryhit iscategory_idin a filter example, anddrivers.mdx's oneNULL-safehit is about platform-table index migrations, not this classification.content/docs/releases/v17.mdxis release-owned and read-only. Checked anyway: itscategoryhits are alltool.category, a retired spec key. Nothing to escalate, and nothing edited there.content/docs/deployment/cli.mdxwas falsified, in the page an operator reads before running the command. Three cells changed, table not restructured:saferow listed "create a declared index" flat, promising auto-apply for exactly the case this branch now blocks — carved out to theUNIQUEsub-case;create_index's row said only that the index is missing, and now carries the pre-flight sentence, matched torecreate_index's existing wording one row down ("block the op with a report instead of failing a boot") rather than given a new shape — the doc edit is the same parity edit as the code;destructiverow gains the counterpart example, because that row is where an operator looks to find out why the op they expected to be safe is not.Clause ② — enumerated, not assumed
The standing check is "doesn't touch
packages/spec" is not "doesn't widen the public surface" — enumerate every published package the surface touches and diff its entry point.@objectstack/driver-sql. Itsfilesshipsdistonly, so the two test files are not published;.changeset/*.mdandcontent/docs/**release nothing.packages/drivers/driver-sql/dist/index.d.ts, built from this branch and fromorigin/main6ed4b811aand diffed.SqlDriver's.d.tscarries its protected members, so this was a real risk — which is why the row formatter is a module-local function rather than a method.diffclean. The only.d.tsdelta is JSDoc prose carried through by tsup. The filter that produced that empty result was fired against a deliberately re-signed copy of the same file first, so the emptiness is a measurement and not a dead filter.⇒ Clause ② stays
no. No published symbol added, removed or re-signed. Draft PR, noneeds:contract-review.What a consumer of the classification now sees
Measured by reading every in-repo consumer of
ManagedDriftEntry.category, not assumed:os migrate plan/os diff(groupByCategory,renderPlan)os migrate apply(intended/deferred)--allow-destructivereports it skipped with no index createdartifact-boot-migration.ts)autoMigrate: 'safe'(reconcileAndWarnDrift)os migrate plandoes not set a non-zero exit on destructive entries, so no exit status changes. Clean data is entirely unaffected: the probe finds nothing, the index is created exactly as before, and the pre-flight adds one groupedSELECTonly for an op that would create a unique index the database does not have.Verification
Union re-run on the final commit —
git rev-parse --short HEAD=2721a7e2b(origin/mainmerged in twice as this landed).pnpm --filter @objectstack/driver-sql exec vitest run— 154 passed, 9 skipped, 2357 tests passed, 0 failed.pnpm --filter @objectstack/driver-sql typecheck— exit 0, andtsc --listFilesconfirms all three edited source files are in that program (1 hit each), so "typecheck is clean" actually covers the new tests.node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack --commandsderived 73 families once the docs edit landed — up from 46 before it, which is exactly why the derivation was re-run against the real diff rather than trusted from the first pass. Asserted against the Reconciliation line's 73. All 73 run at2721a7e2b: 73/73 exit 0, each exit code captured before any pipe. Two of them (check:dual-build-cjs-loads,check:type-check-debt) first answered exit 3 = PREREQUISITE NOT MET = NOT MEASURED; the closure was built and they were re-run to a real green.pnpm check:durability-log-level— exit 0, run although the derivation does not name it, because this diff adds alogDurabilityFailurecall site: "29 durability-critical catch seam(s), all loud".pnpm lint— the full repo-wideeslint . --no-inline-config, exit 0. No narrowing claimed and none needed.sql-driver.tsreverted toorigin/main's blob (mutation confirmed on disk bygit hash-objectequality with the base blob and inequality with the pre-mutation hash, plus marker counts falling to 0 against a firing control), the package rebuilt on both legs, 7 of the new assertions turn RED, then restored —git diff HEADempty, on-disk hash equal to the HEAD blob, rebuiltdist/index.d.tsbyte-identical to the pre-ablation build. The suite resolves the driver through../src/index.js, and the ablation is itself the proof: source-only edits changed the verdict with no rebuild in between.sqlite (
better-sqlite3) is a real driver, not a double — the duplicate rows, the failingCREATE UNIQUE INDEXand the row report are all measured against a live database. ⛔ Nothing pins the database's raw error text: thecatchshape is dialect-independent, that text is not.Known gap, deliberately not fixed here and filed as #15479: the MySQL hash-shadow arm of the same
catch(#11627 / #12998) still guards onnullSafe.size > 0, so a plain unique whose shadowALTERfinds duplicates keeps taking the boot down. It is the same defect class, but it is reachable only on a live MySQL with a key part over the 768-char ceiling, and this container has none — an unmeasurable change to a durability path is worse than a named gap.