docs(devx): measure the verify-lock gate-routing trade — four quantities, no routing

[os-project-manager]

Fixes #12795

Delivers the four quantities scripts/pm/os-verify-lock.sh names as unmeasured in its
WHAT THIS LOCK DOES NOT COVER header, and retires that sentence with a pointer to the result.

⛔ No routing is implemented and nothing about what the lock serialises changed. The routing
decision this feeds is a separate, maintainer-facing card, per the triage grading on #12795.

Diff: 2 files. A new audit under docs/audits/, and a comment-only edit to the lock script's
header. No production code.

---

Answer in one line

Every routing policy measured is worse than routing nothing, on the quantity that decides it.
Today 2.0% of locked runs return exit 99 (a NOT MEASURED run). Across a 64-cell sweep of four
routing policies, four demand levels and four levels of CPU credit handed to routing, the best
cell is 4.0% and the worst is 65.7% — no cell reaches today's 2.0%.

⚠️ Shared-box seconds throughout: 4 cores, 3-4 sibling dev agents live, ambient load 1.2 to 19.6
across the session. Ratios are the durable part, exactly as the header prescribes.

---

1 — Gate-run cost distribution · MEASURED

All 116 root check:* families, each timed as pnpm check:NAME, exit code captured before any
pipe. Two hit the harness timeout and were re-run uncensored.

Whole farm serially: 1153 s. p50 = 2.3 s, p90 = 9.5 s, p95 = 23.1 s, max = 305.5 s.

bucket	families	gate-seconds	% of cost
under 1 s	6	5.8 s	0.5%
1-5 s	79	160.7 s	13.9%
5-30 s	26	245.8 s	21.3%
30-100 s	3	206.4 s	17.9%
over 100 s	2	534.5 s	46.3%

Is the threshold clean? Partly — and the honest answer has two halves. 5 families (4.3%) carry
64.3% of all gate-seconds; the top 2 alone carry 46.3%. Ranks 1-5 are separated by steps of
x1.33, x2.33, x1.41, x1.83, and rank 5 stands x1.65 clear of rank 6. Below rank 6 the
distribution is smooth — every consecutive gap is x1.30 or less. So a cut exists at roughly
30 s, isolating exactly 5 families, and nothing resembling a threshold exists below it.

family	cost
check:pm-dispatch-gates	305.5 s
check:query-options-erasure	229.0 s
check:slot-lookup	98.3 s
check:stall-guard	69.9 s
check:engine-double-contract	38.2 s

⚠️ Correction: "most are sub-second" is false as invoked, true as scripted

Only 6 of 116 families finish under a second and the fastest is 934 ms — because for the light
families roughly 97% of the measured wall time is pnpm's own startup, not gate work:

	time
pnpm check:node-version (3 reps)	966 / 989 / 971 ms
node scripts/check-node-version.mjs — same gate, no pnpm	101 ms
node -e 'process.exit(0)' — control	39 ms

Both readings are true of different things; routing needs the first, because what would be routed
is the command an agent types. The farm count has also moved: dispatch-gates.mjs now discovers
190 families across 29 workflow files, against the card's 176.

2 — Queue-depth effect · MEASURED, and the sign is settled

One fixed CPU-bound gate at W = 1, 2, 4, 8 concurrent copies. W = 1 is the routed world.

sweep	ambient load	W=1	W=2	W=4	W=8
check:objectql-double-limit rep 1	2.0-3.1	4.12	7.31	9.80	7.57
check:adr-anchors	8.5-11.3	9.40	9.27	9.28	10.13
check:objectql-double-limit rep 2	11.6-19.6	4.34	5.94	5.26	6.64

(gates per minute; bold = best in row)

W = 1 is never the throughput maximum, in three sweeps out of three. On an unsaturated box
serialising costs 2.38x aggregate gate throughput; on an already-saturated box the curve is flat
and it costs approximately nothing. Even 2x over-subscription beat serialisation by 1.53x-1.84x.
The card's open possibility — that serialised-but-uncontended beats parallel-but-thrashing — did
not occur at any concurrency tested.

⛔ Per-agent end-to-end wall time is NOT MEASURED: nothing records when a card starts or ends
and no ledger field carries an agent identity. Aggregate throughput at the box bounds it.

3 — Budget fit · MEASURED, and it is decisive

Budget is 540 s (HARD_CAP_S). Baseline from the real ledger, 99 records over 16.1 h:

utilisation	rho = 23.5% · 6.14 runs/h · mean service 148 s
wait, over runs that waited at all	n=26, p50 151 s, p90 536 s, max 540 s
queue-timeouts (exit 99)	2 of 99 = 2.0%

Two facts decide most of this before any model: the wait distribution is already flat against the
cap (p90 536 s of a 540 s budget — no headroom to spend), and the lock is idle 76.5% of the
time and still times out 2% of runs — the load is bursty and heavy-tailed, the regime where added
work is punished super-linearly.

The arithmetic that needs no model

The two heaviest families cost 305.5 + 229.0 = 534.5 s. The budget is 540 s.

Routing just the top two families means one gate sweep can occupy 99.0% of another caller's
entire acquisition budget, by itself, with the lock otherwise empty.

Trace-driven simulation

arrival = ts - held - waited reconstructs each run exactly. Replaying through a FIFO server with
the 540 s budget reproduces the record — exit 99: 2 simulated vs 2 observed; wait p50 151 s vs
151 s; p90 536 s vs 536 s. The residual error direction is known and favours routing: the trace
cannot contain holders it never recorded, so every routed figure is a lower bound on the harm.

Injecting gate sweeps at rate G. f is the build service-time multiplier once gates no longer run
alongside a holder — the credit routing gets for freed CPU. f = 0.60 is more credit than the
concurrency sweep supports. Build exit-99 rate; today's value is 2.0%:

policy	added rho	f=1.00	f=0.85	f=0.70	f=0.60
route ALL 116, G=1	32%	13.1%	10.1%	7.1%	6.1%
route ALL 116, G=2	63%	23.2%	20.2%	14.1%	8.1%
route ALL 116, G=4	129%	45.5%	42.4%	38.4%	40.4%
route >= 10 s (11), G=1	23%	8.1%	8.1%	9.1%	7.1%
route >= 10 s (11), G=4	94%	40.4%	35.4%	38.4%	36.4%
route >= 30 s (5), G=1	20%	8.1%	7.1%	7.1%	4.0%
route >= 30 s (5), G=4	83%	36.4%	29.3%	30.3%	29.3%
route >= 60 s (4), G=1	19%	8.1%	5.1%	7.1%	4.0%
route >= 60 s (4), G=4	79%	33.3%	36.4%	29.3%	31.3%

No cell reaches the 2.0% baseline. The minimum over all 64 cells is 4.0% — double today's rate
— and only at the lowest demand, narrowest policy, and more CPU credit than measured.

A real derived family for an ordinary card is 7 families (measured on this PR's own diff), 16
once a changeset exists — not 116. Plausible fleet demand lands at added rho ~7-25%, the G=1 to
G=2 rows, i.e. 2x to 12x today's exit-99 rate.

⛔ Fleet gate arrival rate is NOT MEASURED and is unmeasurable with this instrument — gate runs
are unlocked, take no ticket and write no ledger row, so the ledger is blind to exactly the
population routing would add. G is swept rather than fitted; the conclusion is its robustness
across the sweep.

4 — The partial option · MEASURED. Wins its bracket, loses the question.

Routing only the heavy families does dominate routing everything, monotonically (at G=4, f=1.00:
45.5% then 40.4% then 36.4% then 33.3%). But the gain is small, for a structural reason:

policy	families	seconds/sweep	% of farm cost
route ALL	116	1153 s	100%
route >= 30 s	5	741 s	64.3%
route >= 60 s	4	703 s	61.0%

Dropping 112 of 116 families removes only 39% of the routed load, because the heavy families
are the load. There is no "route only the expensive ones" that is also "route only a little"
— the threshold from (1) selects precisely the families whose service times are a large fraction
of the whole 540 s budget.

Its one genuine merit is orthogonal to CPU: those 5 families are the ones whose unlocked runs most
distort a concurrent holder's timings, so routing them buys measurement hygiene while costing
measurement availability. That trade is a judgement, not a number.

---

Corrections to inherited premises

• The disclosure half ([finding] os-verify-lock serialises locked heavy jobs but NOT gate scripts — so every timing card measures a contended box while believing the lock gave it an idle one #12528) shipped and the throughput half was open — re-verified on
  origin/main before starting. Premise holds.
• The wait budget was NOT raised. HARD_CAP_S and DEFAULT_WAIT_S are both still 540, and the
  script argues at length that this constant cannot be raised — the budget is spent inside one
  foreground turn a ~600 s ceiling kills. What [finding] os-verify-lock's 540s wait budget is shorter than a single legitimate hold — a compliant caller times out systematically, not occasionally #12538 moved is SLOT_MAX_AGE_S (now
  HARD_CAP_S * 3 = 1620 s), re-based on last relinquish. Any reasoning assuming a larger
  acquisition budget is wrong in the other direction.
• The lock is not the expensive part. Acquire-plus-release on a free lock: ~90 ms (4 reps:
  88, 93, 93, 94 ms) — an order of magnitude cheaper than the ~970 ms pnpm startup every routed
  gate pays anyway. A fifth rep taken first cost 58 106 ms because a sibling held the lock; that is
  contention, not mechanism.
• The --report ledger's /tmp residence ([finding] The os-verify-lock ledger lives in the container's /tmp, so --report can never answer the question it was built for — "where does hold time go" is scoped to one shift, and says so nowhere #12783) bounds this: 16.1 h across 2 boots, one
  container. Stated in the audit rather than left to pass silently.

Verification

19-family derived gate set (node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack,
re-derived on the merged tree) plus the always-runs os-verify-lock.sh --self-test that covers the
edited file. Run at 30bc3e773.

All green except one, which self-declares its own non-result: check-test-completeness.mjs
exits 3 with "no local log to hand it, so the local reading for this gate is NOT MEASURED" and
instructs exactly that when run locally. check:nul-bytes green, plus a counter-checked self-scan
of both edited files (scanner proven live against a control-byte probe in the same command).

skip-changeset: the diff publishes nothing from any package — one file under docs/audits/ and a
comment in scripts/pm/.

---

Generated by Claude Code