perf(datasource): size the primary SQL pool from OS_DATABASE_POOL_MAX - #14776
Conversation
`buildSqlPool` gives every postgres/mysql datasource that declares no `pool`
an explicit `{min:0,max:5}`. The primary datasource — the one behind
`OS_DATABASE_URL` — is composed as a url and nothing else, so that `max: 5`
was the per-replica ceiling on every self-hosted deployment with no operator
knob for it. A driver-level env read would have been dead code behind this
function's explicit object.
Precedence: a declared `pool.max` > `OS_DATABASE_POOL_MAX` > today's `5`.
With the env unset nothing changes, which is the upgrade path for every
existing deployment and is pinned as such. A non-integer value refuses the
boot naming the variable, the value and the sizing rule.
Only the postgres/mysql arms call `buildSqlPool`, so the unsupported arms
(`memory` / `sqlite` / `sqlite-wasm` / `turso`) structurally cannot see it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AUF1NoViznQK32gqpK8wS8
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AUF1NoViznQK32gqpK8wS8
📓 Docs Drift CheckThis PR changes 1 package(s): 1 hand-written doc(s) NAME something this change touched and may need an implementation-accuracy re-verification:
What this run could not see
Coarse fallback — 1 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): Which tree this was computed onThis run read A worktree cut from an older # while this PR is open — GitHub drops the merge commit once it closes
git fetch origin d3d5c70ab838c5a0cf444f6b6a7cd77e247124e3 && git checkout d3d5c70ab838c5a0cf444f6b6a7cd77e247124e3
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin df657d9df1fac5a2222f183008532613a1df585f f82e7ada1f054467656ed84dfed9af37fc9a2012 && git checkout -B drift-repro df657d9df1fac5a2222f183008532613a1df585f && git merge --no-ff f82e7ada1f054467656ed84dfed9af37fc9a2012
node scripts/docs-audit/affected-docs.mjs --json df657d9df1fac5a2222f183008532613a1df585f
|
Queue ejection — the repo-wide signature, first hit on this PR. Re-arming once.The failure. Queue build 33721164982, job This PR's own checks are green — 42 check runs on head Why I read it as not this PR's — and what that reading does not rest on. It does not rest on "the failing test is in What the reading rests on instead:
Anchor: #14822, reopened at 06:0xZ after being closed as a duplicate of a card that is itself closed. The triage workflow has re-adopted it, so this signature has an open home again and the evidence lives there rather than in scattered PR comments. Action: auto-merge re-armed — one re-queue. That is the single discriminating attempt this failure class gets on this PR. If the same signature ejects it again, that is data rather than noise: it goes to #14822, not back into the queue, and this PR waits there. Generated by Claude Code |
Ejected again — and I owe a correction before I re-arm, because I wrote two contradictory rules within half an hourQueue build 33724235522 ejected this PR at 07:01Z on the same
Both are PRs I own. The #14528 rule is the one that survives scrutiny, so it governs here too. Why re-arming is close to cost-free, which is the part the triage comment's "don't re-queue until it's fixed" cannot see. That advice protects the shared queue, and it would be right if each attempt made the break more likely to fire. It does not. #14822's own table shows 28 ejected PRs across only 4 speculation stacks — the flake fires about four times and takes whole batches with it. A PR's presence in a batch does not change how often it fires; it only decides whether that PR is aboard when a batch succeeds. And batches are succeeding: #14804/#14790/#14799 at 04:51:09Z, and this seat's own #14813/#14815 at 05:34:30Z, through the same queue on the same night. The standing rule for this seat's PRs, stated once so it stops drifting between comments: while a PR of mine is green, mergeable and unarmed, it gets armed. Ejections on a repo-wide signature are the queue's defect and are tracked on their anchor — never a reason to leave my own finished PR unable to land. Evidence added to the anchor rather than argued here. #14822 now carries a within-run, same-runner control from another PR's shard: the same child at 7 295 ms and 180 103 ms on the same machine minutes apart (24.7×), which holds load constant by construction and shows the child stalls rather than runs slow. Two ceilings have now been tried (40 s, then 180 s via #14715) and neither closed it. This PR's own 42 checks were green on head Generated by Claude Code |
Fixes #14176
Adds
OS_DATABASE_POOL_MAX, the operator-facing ceiling for apostgres/mysqldatasource's knex pool, read inbuildSqlPool.The card's root cause is falsified; the defect is real
The card blames
SqlDriver.withConnectBoundfor setting no pool size, so knex's{min:2,max:10}default applies. Measured onorigin/main, that is not what the primary datasource runs:buildSqlPool(packages/services/service-datasource/src/default-datasource-driver-factory.ts) hands everypostgres/mysqldatasource an explicit{min: 0, max: 5}unless it declares its ownpoolblock.OS_DATABASE_URL— is composed by the CLI asconfig: { url, ...autoMigrate }with nopool(packages/cli/src/utils/storage-driver.ts, thepostgresarm).So the effective per-replica ceiling was 5, not 10, and
SqlDrivernever sees an "unspecified" pool from this path — an env read in the driver would have been dead code behind the factory's explicit object. The card's own connection counts corroborate 5 over 10: 3 replicas x 5 + admin/sampler is about the ~21 observed; three pools saturated at 10 would have shown ~30.This matches the correction already recorded on the issue and the maintainer ruling of 2026-09-02 (option A).
What changed
buildSqlPoolnow resolvesmaxas declaredpool.max>OS_DATABASE_POOL_MAX>5. It is the only site that decides the unspecified case, so the precedence is expressed once, and the driver needs no knowledge of the factory's choice.{min: 0, max: 5}, byte-identical to today — that is the upgrade path for every existing deployment, and it is pinned by a test whose job is to go red if it ever drifts. A blank value reads as unset, so a declared-but-unfilled compose variable also keeps today's behaviour.Number(process.env.X ?? default)shape, where a typo becomesNaNand the operator trying to raise the ceiling silently keeps the one they meant to leave. A pool ceiling is only ever measured in production.OS_DATABASE_POOL_MINis not exposed (ruling: this path already runsmin: 0; a later patch if ever needed).OS_DATABASE_*per AGENTS.md Prime Directive 9 —DATABASEis the existing family (OS_DATABASE_URL,OS_DATABASE_DRIVER,OS_DATABASE_SQLITE_JOURNAL_MODE);OS_DB_*has zero hits in the repo.What I measured, and what I did not
503rate come from a live 3-replica EE cluster on 2026-09-01. There is no cluster and no live database here, and a fabricated local rerun would be evidence of nothing. I did not re-measure them. What is pinned instead is the mechanism those numbers rest on: which pool size actually reaches knex.Measured here:
@objectstack/service-datasourcesuite: 0 failed | 635 passed without the new pin file, 0 failed | 643 passed (31 files) with it, atf82e7ada1.tsc --noEmitexit 0, with--listFilesconfirming both the changed source and the new test are in the program (this package's tsconfig includessrc, tests and all).scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack): 59 commands, 55 exit 0. The 4 non-zero are all NOT MEASURED by their own verdict text, from unbuilt packages outside this diff's closure —check-test-completeness("PREREQUISITE NOT MET ... ⛔ It is not a red"),check:dual-build-cjs-loads("⛔ This is NOT a pass: nothing was measured"),check:type-check-debt("⛔ This is NOT a pass and NOT a finding"), andcheck:skill-examples, which refuses becausepackages/client-react/distis unbuilt and never reaches the docs surface.pnpm lint(eslint . --no-inline-config): exit 0, full population, no narrowing.Ablation — the knob is load-bearing. With the change committed,
buildSqlPoolwas mutated to drop the env read (marker injected as aglobalThisproperty, not a comment). The mutation was confirmed on disk before running: deleted-text occurrences 1 to 0, injected marker 1, blob5efad548to47abe72b. The pins then read 4 failed | 4 passed: the four knob pins went red, while the unset-default pin, the blank-value pin, the declared-pool-wins pin and the unsupported-arm pin stayed green — the predicted direction, since none of those four depends on the env wiring. Restore proved by blob equality withHEAD(5efad548), emptygit diff HEAD, and zero leftover markers.Resolution path: the tests import the factory by relative path, so they read TypeScript source, not
dist— no rebuild leg applies, and the ablation moving the result with no rebuild in between demonstrates it.Not touched, deliberately
POOL_UNSUPPORTED_DRIVER_IDSand the authoring-rejection path are untouched.memory/sqlite/sqlite-wasm/tursoreject a declaredpoolunder three maintainer rulings, and knex's better-sqlite3 dialect pins{min:1,max:1}on purpose. The env is read insidebuildSqlPool, which only thepostgres/mysqlarms call, so those arms structurally cannot see it — pinned by a test rather than asserted.packages/drivers/**andpackages/spec/**are untouched (read only).Clause-2: yes
Derived from the diff, not recalled:
git diff -U0 origin/main...HEAD | grep -E '^\+\s*export 'returns nothing (zero new exported symbols), and no declared spec key is added (packages/spec/**untouched). The change reads env inside an existing non-exported function.Those mechanical indicators point
no, and I am reporting them because they are real information for the reviewer — but the grading is still yes: the widening is the documented operator-facing environment variable itself, a permanent public configuration obligation, which is exactly what the maintainer already gradedyeswhen ruling option A. A mechanical export-grep should not argue down a ruling that considered this precise change, andnowould have been the convenient answer here rather than the right one.Changeset:
minoron@objectstack/service-datasource— a new operator capability, backward-compatible, no default moved.🤖 Generated with Claude Code
https://claude.ai/code/session_01AUF1NoViznQK32gqpK8wS8
Generated by Claude Code