Skip to content

fix(tooling): raise the re-measure heap ceiling on a runner measurement, not on a typed number - #14805

Draft
claude[bot] wants to merge 4 commits into
mainfrom
claude/issue-14569-tsc-heap-ceiling-6144
Draft

fix(tooling): raise the re-measure heap ceiling on a runner measurement, not on a typed number#14805
claude[bot] wants to merge 4 commits into
mainfrom
claude/issue-14569-tsc-heap-ceiling-6144

Conversation

@claude

@claude claude Bot commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Fixes #14569

The ruling was "A, with the measurement first". This PR carries the measurement, and the measurement is why the constant does not move: CI_TSC_HEAP_CEILING_MB describes V8's default old space on the Type Check · debt ledger runner, that default is 4096 MB, and a pin above it is refused by the gate itself. The raise needs one thing this constant cannot carry — see "What the raise still needs" below, which is the open decision.

How the reading was taken

Job logs cannot be downloaded from the agent container, but check-run annotations are readable over REST. So the first commit on this branch is a TEMPORARY probe — a --runner-reading mode plus one step in the typecheck-debt job — that emitted its numbers as ::notice workflow commands. It ran in run 33708954003 / job 100504131338 and was read back through GET /repos/objectstack-ai/objectstack/check-runs/100504131338/annotations. The second commit reverts it in full: the tree is byte-identical to main apart from the comment.

Measured on the runner — 2026-09-03, image ubuntu24 20260831.293.1, Node v22.23.2, 4 vCPU

Reading Value
MemTotal 16,373,452 kB (~15.6 GiB) — not the 7 GB the finding assumed
SwapTotal 3,145,724 kB
MemAvailable when the re-measure starts 14,329,064 kB
Other consumers in the job 153 processes holding 940,316 kB (~918 MB): Runner.Worker 144 MB, Runner.Listener 98 MB, provjobd 96 MB, dockerd 73 MB, containerd 43 MB
Concurrency the job's steps are sequential — nothing in it runs beside the re-measure
Gate process heap_size_limit, NODE_OPTIONS unset 4144 MB, i.e. a 4096 MB old space
A child under --max-old-space-size=6144 heap_size_limit 6192 MB

packages/qa/http-conformance's TEST_DEBT program — the heaviest one, 906 files, 692,003 lines of definitions, 7,328,937 instantiations — run twice with --extendedDiagnostics:

Cap tsc "Memory used" Peak RSS Check time OOM
--max-old-space-size=4096 4,077,718K 4,212,904 kB 26.84 s no
--max-old-space-size=6144 4,420,706K 4,545,500 kB 21.90 s no

Lowest MemAvailable seen at any point during either run: 10,562,192 kB.

That pair is the headroom reading the finding asked for, and it confirms the finding's premise: handed 343 MB more heap the same program keeps 343 MB more live and finishes ~5 s sooner, so under 4096 it is paying GC pressure to fit rather than fitting.

Why the constant does not move in this PR

The scarce resource is not the runner's memory — 15.6 GiB with ~918 MB spoken for, and 10.1 GiB still free at the tightest moment of the heaviest program. It is V8's default old space on that runner, which is 4096 MB. This constant describes that default, and as of 2026-09-03 it still describes it exactly.

Raising it alone cannot buy the ledger a roomier run, and that is measured rather than reasoned:

  • remeasureHeapCeiling takes the minimum of the pin and the limit the running process actually has. With the pin at 6144 and the gate started under the runner's own default, the chosen ceiling is still 4144.
  • The stale arm then refuses the run outright. Reproduced locally against a 4144 MB process (GITHUB_ACTIONS=true NODE_OPTIONS=--max-old-space-size=4096, pin temporarily set to 6144): --re-measure exits 1 before the first tsc, printing "the pin is now ABOVE the ceiling it claims to describe". Shipping the bare raise would turn the Type Check · debt ledger lane red on every PR and on main.

So the ruling's execution scope — the constant, its comment, and the self-test rows — is sufficient for the evidence half and not for the raise half. The evidence half lands here; the raise is escalated rather than guessed.

What the raise still needs (open decision on #14569)

Option What it costs
A1 Give the gate process the memory — NODE_OPTIONS: --max-old-space-size=6144 on the re-measure step in lint.yml — then pin 6144 One workflow line plus the constant. Every invariant survives: the pin still never exceeds what the process really has, and the existing stale arm keeps the two numbers honest, loudly, on the runner. Outside the ruled file surface, which is why it is not taken here
A2 Change the cap rule: derive the second candidate from the box's real memory instead of this process's V8 default Architecturally the more correct fix — the candidate exists to stop us promising memory the box does not have, and today it approximates that with V8's default, which is wrong on any big box. But it inverts the "never RAISE" doctrine the file's header argues at length, and needs new adversarial self-test rows
A3 Do not raise; take option C from the finding (narrow http-conformance's imports) Leaves the ceiling describing the runner exactly, which it does. Recurs for the next heavy entry

Recommendation: A1. It is the smallest change that makes the pin mean what the ruling wants it to mean, it keeps the pin a description of the process rather than a wish, and the stale refusal already exists to catch the two numbers drifting apart. A2 is the better shape long-term but is a policy inversion that wants its own decision, not a rider on this one.

What lands here

scripts/check-type-check-coverage.mjs only — 63 changed lines, all of them comment lines (mechanically counted: git diff -U0 | grep '^[+-]' gives 63 lines, 0 of which are non-comment):

  • the constant's comment gains the first-hand runner reading above, replacing archaeology through a failed job's GC trace as the pin's provenance;
  • the existing "if 4096 is wrong, it is wrong DOWNWARD" paragraph is kept and extended with the note that the new reading lands on the same 4096 from the other side;
  • the existing "do not raise this" paragraph is kept and gains the measured mechanism above;
  • the self-test row for "a box shaped like CI" gains a note that its + 48 is now the measured runner (4144 MB) rather than a construction.
 scripts/check-type-check-coverage.mjs | 63 ++++++++++++++++++++++++++++++++++-
 1 file changed, 62 insertions(+), 1 deletion(-)

Verification, at 24ee34f8d7

node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack --commands derived 19 families for this diff; 17 ran green, one is NOT MEASURED by construction, and one was narrowed:

  • green: check:agent-test-spelling, check:bash32-floor, check:cli-command-ids, check:cross-package-test-inputs, check:entry-guard, check:parse-guard, check:pm-dispatch-gates, check:pnpm-filter-targets, check:ratchet-remedy-authority, check:type-check-coverage, check:watch-hint-literal, check-ci-filter-parity, check-cross-package-test-inputs, check-self-test-wired, check-shard-attestation, check-whole-set-label-write, bare-root-worklist --self-test.
  • node scripts/check-test-completeness.mjs exits 3 — its own "nothing was measured" code, because no vitest summary exists locally. Not a finding.
  • pnpm check:type-check-debt was narrowed, declared: it needs a full closure build plus ~4 min of 34 sequential tsc programs on a shared container, and the diff is comment-only (63/63 changed lines are //). The substitutes: --self-test is green (43 re-measure cases, including all seven remeasureHeapCeiling rows), check:type-check-coverage is green, and the debt lane itself ran green in CI on this branch at d415f4fa40 (job 100504131338) on a tree that carried more than this one.

Serial constraint: #14785 merged at 02:52:13Z and is merged into this branch. #14627 is still open and also touches this file (ledger rows, not the constant).

🤖 Generated with Claude Code

https://claude.ai/code/session_01WLJQhde67SeTccsmnBVarV


Generated by Claude Code

#14569)

Reverted before this PR's final diff. `CI_TSC_HEAP_CEILING_MB` may only move
on a reading taken where the verdict is taken -- the `Type Check · debt ledger`
job on `ubuntu-latest` -- and this container cannot download job logs. Check-run
ANNOTATIONS are readable over REST, so the probe emits its readings as
`::notice` workflow commands from that job:

- the runner's MemTotal/MemAvailable/Swap, image, nproc, and the gate process's
  own V8 `heap_size_limit` (the runner's default old space);
- what else is resident at the point the re-measure starts (`ps` RSS census);
- the `packages/qa/http-conformance` TEST_DEBT program -- the same generated
  project `measureTestDebt` writes -- run with `--extendedDiagnostics` under
  `--max-old-space-size=4096` and under `6144`, reporting tsc's own "Memory
  used", peak RSS, and the machine's minimum MemAvailable during each run.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WLJQhde67SeTccsmnBVarV
@claude claude Bot added the skip-changeset PR has no user-facing published change; bypasses the changeset gate label Sep 3, 2026
…NG_MB

The pin's provenance was archaeology through a failed job's GC trace, which
bracketed the runner's old space into [4040, 4148] MB. It is now a first-hand
reading, taken where the verdict is taken -- inside the `Type Check · debt
ledger` job, by a temporary probe step (reverted in the previous commit) that
emitted its numbers as `::notice` annotations:

  runner        ubuntu24 20260831.293.1, 4 vCPU, MemTotal 16,373,452 kB
                (~15.6 GiB) -- not the 7 GB the finding assumed
  consumers     153 processes / 940,316 kB (~918 MB); the job's steps are
                sequential, so nothing runs beside the re-measure
  this gate     heap_size_limit 4144 MB with NODE_OPTIONS unset -- the runner's
                V8 default, confirming the 4096 MB old space directly
  heaviest      qa/http-conformance's TEST_DEBT program under two caps:
  program       4096 -> 4,077,718K used, 4,212,904 kB peak RSS, 26.84s check
                6144 -> 4,420,706K used, 4,545,500 kB peak RSS, 21.90s check

The pair is the headroom reading the finding asked for: 343 MB more heap keeps
343 MB more live and finishes ~5s sooner, so under 4096 the program is paying
GC pressure to fit. The constant does NOT move on it, and the measurement is
why: the scarce resource is V8's default old space (4096 MB), not the runner's
memory, and this number describes that default exactly.

The comment also records what the measurement made mechanically visible --
raising the pin alone cannot deliver a roomier run. `remeasureHeapCeiling`
minimises over the pin and the running process's own limit, so a 6144 pin under
the runner's default still chooses 4144, and the `stale` arm then refuses the
run outright: `--re-measure` exits 1 before the first tsc. Reproduced against a
4144 MB process. Delivering a raise needs the gate PROCESS given the memory
first, which is a workflow decision and is escalated on #14569.

The self-test row for "a box shaped like CI" gains a note that its `+ 48` is
now the measured runner rather than a construction.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WLJQhde67SeTccsmnBVarV
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci/cd size/s skip-changeset PR has no user-facing published change; bypasses the changeset gate

Projects

None yet

1 participant