plan #944: context-window fold budget (A3, source #551)

[btipling]

Plan #944 — context-window fold budget (A3, source #551)

Implements the A3 fold-budget policy: the durable-turn LLM context is sized to the selected model's context window minus a completion reserve, measured in tokens, replacing the CRUD-style caps that governed it (maxMessages 400 + maxChars 3.5M char tail). Policy + accounting slice only — occupancy metering (#556) and compaction (#552 / A4) remain separate boards.

Caps (6 NEW in lib/sessionCloudCaps.ts — no existing cap value changed)

Cap	Value	Role
CONTEXT_WINDOW_DEFAULT_TOKENS	200 000	Conservative default when neither catalog source publishes a window — never a fabricated window
CONTEXT_RESERVE_MIN_TOKENS	16 384	Pi-style reserve floor
CONTEXT_RESERVE_FRACTION	0.15	Fractional reserve (effective reserve = max(floor, fraction × window))
MODEL_MSG_SEED_MAX_ROWS	4 096	Row-count safety rail (replaces the retired 400 intelligence cap)
MODEL_MSG_SEED_MAX_BYTES	2 MiB	Workflow run-arg carrier bound on the trimmed seed
CONTEXT_CHARS_PER_TOKEN	4	Documented estimator ratio (ceil(chars/4)) — never a tokenizer

Layers

• Window source — lib/gateway/modelCatalog.ts: Gateway context_length + models.dev limit.context parsed into parallel window maps with the same TTL / single-flight / fail-open / negative-cache discipline as the effort maps; getJoinedWindowMap (Gateway wins disagreements, overlay fills holes). /api/models entries carry contextWindow (omitted when unpublished); lib/harnessModelCatalog.ts is the new pure host parse (extracted from HarnessHost.tsx for row-13 testing); lib/agent/contextWindow.ts applies the conservative default.
• Budget math — lib/agent/contextBudget.ts: estimateTokens + foldBudgetTokens(window, modelId).
• Seed trim (durable path) — lib/agent/modelMessages.ts trimModelMessagesToBudget: token budget + row rail + byte rail; drop oldest, re-pair, never drop the newest row. Applied in app/api/turns/route.ts at the route boundary (never inside a 'use step'), on the window resolved pre-start, before start().
• Legacy fold (host) — lib/sessionStore.ts formatPromptWithHistory: token-budget first; the 400-message default is retired to the row rail; maxChars≈3.5M is demoted to a transport backstop; the newest history row and the current ask always survive (a lone oversized ask is sent, never blocked). contextWindow rides RunHarnessChatOptions (lib/harnessChat.ts) from the host's windowByIdRef (app/harness/HarnessHost.tsx).

Out of scope (per plan)

Occupancy metering (#556) and compaction (#552 / A4). No zig changes, no Production mutate.

Tests

New: lib/agent/contextWindow.test.ts, lib/agent/contextBudget.test.ts, lib/gateway/modelWindow.test.ts, lib/harnessModelCatalog.test.ts. Extended: lib/agent/modelMessages.test.ts, lib/sessionStore.test.ts, app/api/turns/route.test.ts, app/api/models/route.test.ts.

Gates: npm run typecheck ✅ · vitest run --changed (86 files / 2 097) ✅ · full suite 178 files / 3 406 tests ✅

Docs: docs/harness-limits.md, docs/session-model.md, docs/agent-stream.md, docs/feature-divide.md, AGENTS.md.

Fixes #944

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment

Project	Deployment	Actions	Updated
invincible	Ignored		Sep 5, 2026 5:46am UTC

[btipling]

Adversarial review — PR #945

Verdict: CONCERNS
Repo: btipling/invincible
Scope: main ← plan/context-window-budget · 24 files · A3 window-derived fold budget (plan #944 / source #551)
Lenses run: L1, L2, L5, L6, L8 (skip L3: host only sizes the legacy promptHistory fold; durable seed is server-side, no dual-chat; skip L4: no workflow/runner/wasm; skip L7: caps are named config, not a single-tenant bind; skip L9: no palette/UX chrome)
AGENTS.md read: yes · docs/feature-divide.md read (agent-loop + host fold)

Findings

Sev	Lens	Finding	Break scenario	Refutation attempt	Confidence
Major	L1+L6	trimModelMessagesToBudget (lib/agent/modelMessages.ts) accounts a different payload than the model receives. (1) Token estimate sums user.content / assistant.delta.text / tool result-or-error only — assistant.delta.toolCalls[].args and JSON framing are invisible. (2) The route then appends userMessage = parsed.prompt after the trim (turnLoop [...priorMessages, {role:'user'}]). Comments and row-14 tests treat the newest seed row as “the current ask”; it is last-turn history. The host fold (formatPromptWithHistory) does include the ask in estimateTokens. The 2 MiB MODEL_MSG_SEED_MAX_BYTES rail is a Workflow-arg bound, not a window bound (2 MiB ≫ 170k×4 chars).	Default 200k window → budget 170k. Seed of many str_replace rows: tiny delta.text, 8 KiB args × 200 calls ≈ 1.6 MiB uncounted. Token rail is inert; byte rail lets ~2 MiB through to a 200k-token model → provider context_length_exceeded. Same with a seed already trimmed to 170k plus a 256 KiB current ask (~64k tokens) against a 30k reserve that was meant for completion + system/tools, not the ask.	Defender: “reserve is 15%/16k for completion + overhead; oversized ask is sent anyway (row 14).” Reserve is Pi-style completion headroom. The host path already puts the ask inside the budget and drops history to fit. Durable production path is the pointer seed; tests (route.test.ts row 7) even assert priorMessages: [newest seed] and userMessage: 'continue' as two payloads and never estimate the sum. Args are persisted verbatim (buildModelMessages does not truncate them).	high
Minor	L5	getJoinedWindowMap re-GETs Gateway /v1/models and models.dev independently of the effort maps (getGatewayWindowMap / getModelsDevWindowMap vs fetchGatewayEffortMap / overlay effort). /api/models Promise.alls both joins → 4 HTTP calls to 2 URLs on a cold isolate; a turn start that also fetches effort does the same.	Cold GET /api/models or first POST /api/turns after TTL: four catalog round-trips, two of them duplicate bodies, each with GATEWAY_MODELS_FETCH_TIMEOUT_MS. Rate-limit / latency on the turn-start boundary.	Defender: TTL + single-flight per map; fail-open. True after the first success; the duplicate is still on the cold path this PR added, and the payload is already parsed two ways from one JSON shape.	high
Nit	L8	Trim / route comments (trimModelMessagesToBudget header, app/api/turns/route.ts “never drop the newest row (the current ask)”) name the newest seed row as the current ask. userMessage is the current ask; the newest seed row is the previous turn.	A later change “protects the newest row” thinking it is the live prompt, and refuses to drop last-turn history that is what actually overflows.	Comment-only; behavior is the Major above.	high

Residual risk

Chars/4 is a locked English-prose estimator, not a tokenizer — CJK/code will under-trim. Unpublished windows fail-open to CONTEXT_WINDOW_DEFAULT_TOKENS (200k): a small model whose catalogs miss context_length is over-budget, a 400k model is over-trimmed. Reserve does not size persona / skills catalog / working-notes / tool-schema overhead (#556 occupancy is out of scope). #277 still fail-closes stale edits. No live Gateway round-trip in this attack.

Merge guidance

CONCERNS: do not merge until the Major is fixed — the token rail must estimate the serialized seed the model sees (including toolCalls.args) plus the current userMessage, and may drop every seed row to fit the ask (host already does). The 2 MiB rail stays the Workflow-arg bound, not a substitute window. Duplicate catalog GETs should share one payload parse. DI/cost: no new PGlite / live I/O in changed tests — not a gate.

Do not --approve from this skill.

What was not attacked

Live Gateway catalog payloads, prod Workflows canvas, occupancy meter (#556), compaction (#552), Wasm status-bar paint of the window, tokenizer parity vs provider counts.

[btipling]

Adversarial-review follow-up (78cece3)

Landed the CONCERNS Major + the catalog Minor on this branch. Not merging.

Finding	Fix
Major L1+L6 — token rail counted user.content / assistant.delta.text / tool result only, then the route appended userMessage; toolCalls.args and the current ask were invisible, so the 2 MiB Workflow rail could ship a seed larger than the model window	trimModelMessagesToBudget estimates JSON.stringify(seed) + currentUserContent; the route passes parsed.prompt. When the ask is in the estimate, history may trim to [] (host fold already does this). No-ask callers still keep the newest seed row.
Minor L5 — getJoinedWindowMap re-GET Gateway + models.dev independently of the effort maps (4 HTTP to 2 URLs on a cold /api/models)	One GET per source parses both effort and window into a shared bundle cache.
Nit L8 — comments named the newest seed row “the current ask”	Comments + docs/agent-stream.md corrected.

npm run typecheck ✅ · targeted vitest 9 files / 152 tests ✅

[btipling]

Adversarial review — PR #945

Verdict: CONCERNS
Repo: btipling/invincible
Scope: main ← plan/context-window-budget @ 78cece3 · 24 files · A3 window-derived fold budget (plan #944 / source #551), including the prior-review follow-up
Lenses run: L1, L2, L5, L6, L8 (skip L3: host only sizes the legacy promptHistory fold; durable seed is server-side, no dual-chat; skip L4: no workflow/runner/wasm; skip L7: caps are named config, not a single-tenant bind; skip L9: no palette/UX chrome)
AGENTS.md read: yes · docs/feature-divide.md read (agent-loop + host fold)

Prior review on 49d56d7 (CONCERNS: seed token rail missed toolCalls.args + current ask; duplicate catalog GETs) is addressed on 78cece3. This pass attacks the follow-up head.

Findings

Sev	Lens	Finding	Break scenario	Refutation attempt	Confidence
Major	L1+L6	parseContextWindow / parseGatewayWindowMap (lib/gateway/modelCatalog.ts) read Gateway /v1/models context_length. Live list payloads (and the REST schema) publish data[].context_window. context_length lives on the endpoints resource, not the list this catalog GET hits. 2026-09-05 live GET: 373 rows, 0 with context_length, 272 with context_window. Gateway window map is always empty; joinWindowMaps "Gateway wins disagreements" is dead; join is overlay-only. Tests (lib/gateway/modelWindow.test.ts) fixture only context_length, so the suite cannot catch a live-shaped payload.	(1) mistral/codestral: Gateway window 128k, models.dev limit.context 256k. Overlay wins → budget ≈ 217k against a 128k model → provider context_length_exceeded. (2) morph/morph-v3-fast: Gateway 82k, overlay 16k → reserve floor 16384 → budget 1 token, history wiped. (3) A new small-window model on Gateway but not yet in vercel.models gets CONTEXT_WINDOW_DEFAULT_TOKENS 200k and is over-fed. /api/models.contextWindow and the host fold inherit the empty Gateway map.	Defender: "overlay fills 293/373 ids; claude/gpt match; the 80 overlay-misses currently have no Gateway window < 200k." Snapshot coverage is not the join contract. The plan named Gateway as primary. max_tokens on the list is the output cap (often 16k) — must not be a fallback or every model under-trims. Prior-review follow-up did not touch this parser.	high
Minor	L8	AGENTS.md, docs/session-model.md, and formatPromptWithHistory (lib/sessionStore.ts) still say the newest seed/history row always survives. 78cece3 made that false when the current ask is in the estimate (trimModelMessagesToBudget minKeep=0; host fold also drops every history row to keep the ask). docs/agent-stream.md was corrected; the agent-contract table was not.	A later implement-plan agent reads AGENTS.md, "protects the newest row", and reintroduces the prior Major (last-turn history not yielding to userMessage).	Comment/docs only on this pass; behavior is already ask-aware. Incomplete leftover of the last nit.	high

Residual risk

Chars/4 is a locked English-prose estimator, not a tokenizer — CJK/code will under-trim. Even with the right Gateway field, unpublished windows fail-open to 200k: a small model whose catalogs miss a window is over-budget, a 400k model is over-trimmed. Reserve does not size persona / skills catalog / working-notes / tool-schema overhead (#556 occupancy is out of scope). #277 still fail-closes stale edits. No live durable-turn round-trip in this attack.

Merge guidance

CONCERNS: do not merge until the Major is fixed — parseContextWindow must read context_window (list) and may accept context_length as an alias; never max_tokens. Tests must include a live-shaped fixture (context_window present, context_length absent) and a disagreement where Gateway context_window wins over overlay. Living-docs / AGENTS.md must stop claiming the newest seed row always survives. DI/cost: no new PGlite / live I/O in changed tests — not a gate.

Do not --approve from this skill.

What was not attacked

Live Workflows canvas, occupancy meter (#556), compaction (#552), Wasm status-bar paint of the window, tokenizer parity vs provider counts, models.dev limit.context schema drift beyond the sampled dump.

[btipling]

Adversarial-review follow-up (8001c6b)

Landed the CONCERNS Major + the living-docs Minor on this branch. Not merging.

Finding	Fix
Major L1+L6 — parseContextWindow / parseGatewayWindowMap read list-payload context_length; live GET /v1/models publishes context_window (0/373 rows had context_length). Gateway map was always empty; join was overlay-only (mistral/codestral 256k overlay vs 128k Gateway; morph/morph-v3-fast 16k overlay vs 82k Gateway). Tests fixtured the wrong field.	Parse context_window (list); accept context_length as an alias; never max_tokens (output cap). Live-shaped fixture + Gateway-wins-disagreement test.
Minor L8 — AGENTS.md / docs/session-model.md / formatPromptWithHistory still said the newest seed/history row always survives after 78cece3 made that false when the ask is in the estimate	Wording corrected: the current ask always survives; history may trim to empty to fit it.

npm run typecheck ✅ · targeted vitest 8 files / 131 tests ✅

[btipling]

Adversarial review — PR #945

Verdict: CONCERNS
Repo: btipling/invincible
Scope: main ← plan/context-window-budget @ 8001c6b · 24 files · A3 window-derived fold budget (plan #944 / source #551), including both prior-review follow-ups
Lenses run: L1, L2, L5, L6, L8 (skip L3: host only sizes the legacy promptHistory fold; durable seed is server-side, no dual-chat; skip L4: no workflow/runner/wasm; skip L7: caps are named config, not a single-tenant bind; skip L9: no palette/UX chrome)
AGENTS.md read: yes · docs/feature-divide.md read (agent-loop + host fold)

Prior reviews on 49d56d7 / 78cece3 (seed missed toolCalls.args + current ask; Gateway parsed context_length instead of live context_window; duplicate catalog GETs) are addressed on 8001c6b. This pass attacks the current head. Live GET /v1/models (2026-09-05): 373 rows, 355 with context_window, 0 with context_length.

Findings

Sev	Lens	Finding	Break scenario	Refutation attempt	Confidence
Major	L5+L1	trimModelMessagesToBudget (lib/agent/modelMessages.ts) and formatPromptWithHistory (lib/sessionStore.ts) drop one oldest row per iteration and JSON.stringify the remainder every time. That is O(n · seedBytes) on the production POST /api/turns start path (before start()), and the same shape on the host fold after the 400-cap was retired to MODEL_MSG_SEED_MAX_ROWS = 4096.	Session at the existing store rails (MODEL_MSG_CHECKPOINT_MAX_ROWS 4096 / MODEL_MSG_CHECKPOINT_MAX_BYTES 8 MiB — a tool-heavy agent session; per-result excerpt is 2k chars). User hits Send. Route reads the blob, then the token rail (default budget 170k ≈ 680k chars) must drop ~90% of a 5–8 MiB array. Measured on this machine with the same loop: 4096 × ~1.8kB rows → 17s of stringify+shift before start(); 1500 × 2kB (already over the 2 MiB workflow rail) → 2.2s. Tiny-window wipe (budget 1, openai/gpt-3.5-turbo 16385 − 16384 floor) is the same 17s. Host fold does the equivalent on the main thread for roll-forward sessions.	Defender: “real sessions are tens of rows; 17s is only at the cap; Function maxDuration is 1800s.” The 4096 / 8 MiB rails are this PR’s own neighborhood (seed row rail copies the store cap; #936 already allows the blob to fill). A tool-heavy durable session is the product path, not a fuzz input. 17s of CPU on the start isolate is a hung Busy / platform 504 class, not a style nit. Binary search (or a prefix-sum of per-row JSON.stringify) is monotonic — allowed(i) for suffix rows[i:] — and is ~12 stringifies instead of ~3600. Tests only exercise 4-row wipes (route.test.ts row 7, contextWindow: 800), so they cannot catch this.	high

Residual risk

Chars/4 is a locked English-prose estimator — CJK/code under-trims. Unpublished / context_window: 0 rows (83 live Gateway rows are 0, parser correctly omits them) fail-open to 200k. Pi reserve floor 16384 equals a 16k window (openai/gpt-3.5-turbo 16385 → budget 1); history wipe there is coded and tested, not a surprise. Reserve does not size persona / skills / working-notes / tool-schema / in-turn growth (#556 occupancy is out of scope). Seed trim at the route means persist re-derives from already-trimmed loop messages — switching to a larger-window model cannot recover dropped structured prefix (display checkpoint still has the transcript). No live durable-turn round-trip in this attack.

Merge guidance

CONCERNS: do not merge until the Major is fixed — the token/row/byte rails must find the cut with O(log n) serializations (binary search the suffix start, or one-pass per-row sizes + prefix sum), keep the same drop-oldest / minKeep / re-pair contract, and cover it with a many-row test (hundreds+, not 4). Host formatPromptWithHistory needs the same cut, not a while-shift rebuild. DI/cost: no new PGlite / live I/O in changed tests — not a gate.

Do not --approve from this skill.

What was not attacked

Live Workflows canvas, occupancy meter (#556), compaction (#552), Wasm status-bar paint of the window, tokenizer parity vs provider counts, models.dev limit.context schema drift beyond the parser tests.

[btipling]

Adversarial-review follow-up (f3fed8c)

Landed the CONCERNS Major on this branch. Not merging.

Finding	Fix
Major L5+L1 — trimModelMessagesToBudget / formatPromptWithHistory dropped one oldest row per iteration and reserialized the remainder (JSON.stringify / full fold rebuild). At the store rails (4096 × ~2k-char rows) that was ~17s of CPU on POST /api/turns before start().	Binary-search the suffix start (allowed(i) for rows[i:] is monotonic). Same drop-oldest / minKeep / re-pair / current-ask-survives contract. Many-row test asserts < 40 JSON.stringify calls (linear was ~300+). Host fold uses the same keep-count search.

npx tsc --noEmit ✅ · targeted vitest modelMessages + sessionStore + turns/route + contextBudget (108 tests) ✅

[btipling]

Adversarial review — PR #945

Verdict: PASS WITH NOTES
Repo: btipling/invincible
Scope: main ← plan/context-window-budget @ f3fed8c · 24 files · A3 window-derived fold budget (plan #944 / source #551), including three prior-review follow-ups
Lenses run: L1, L2, L5, L6, L8 (skip L3: host only sizes the legacy promptHistory fold; durable seed is server-side, no dual-chat; skip L4: no workflow/runner/wasm; skip L7: caps are named config, not a single-tenant bind; skip L9: no palette/UX chrome)
AGENTS.md read: yes · docs/feature-divide.md read (agent-loop + host fold)

Prior reviews on 49d56d7 / 78cece3 / 8001c6b (seed missed toolCalls.args + current ask; Gateway parsed context_length instead of live context_window; duplicate catalog GETs; linear drop-one stringify) are addressed on f3fed8c. This pass attacks the current head. Live GET /v1/models (2026-09-05): 373 rows, 272 with context_window > 0, 0 with context_length, 83 with context_window: 0 (image/video/embed — parser omits them).

Findings

Attack did not breach with current evidence. No Blocker / Major / Minor survived self-refutation on f3fed8c.

Traced and held:

• trimModelMessagesToBudget estimates JSON.stringify(slice) + currentUserContent; route passes parsed.prompt; minKeep=0 when the ask is in the estimate; rePairModelMessages after the cut.
• Binary search: allowed(i) for suffix rows[i:] is monotonic (row/token/byte); allowed(maxStart) is forced true. Host formatPromptWithHistory uses the dual keep-count search; lo===0 drops HEADER and keeps the ask.
• parseContextWindow reads list context_window, aliases context_length, never max_tokens. joinWindowMaps Gateway-wins is live. One SourceBundle GET per source parses effort + window.
• /api/models omits unpublished windows; host parseWindowValue fail-closes; server contextWindowForModel rejects 0 / frac / non-finite and applies CONTEXT_WINDOW_DEFAULT_TOKENS.
• Window resolve stays at the route boundary (not inside 'use step'). turnWorkflow / modelGenerateStep still must not import modelCatalog (existing static-graph pin).
• Changed tests mock catalog / DI seams; no new PGlite / live I/O.

Residual risk

Chars/4 is a locked English-prose estimator — CJK/code under-trims (a 170k-token budget can admit ~4× too many CJK chars). 83 live Gateway rows with context_window: 0 (and any catalog miss) fail-open to 200k: image/video/embed models granted as chat would be over-fed; a 400k model whose catalogs miss a window is over-trimmed. Pi reserve floor 16384 equals a ~16k window (openai/gpt-3.5-turbo 16385 → budget 1); history wipe there is coded. Reserve does not size persona / skills / working-notes / tool-schema / toModelMessages wrapper / in-turn growth (#556 occupancy is out of scope). MODEL_MSG_SEED_MAX_BYTES 2 MiB caps the Workflow run-arg even when the window would allow more (grok 2M / gpt-5.4 1.05M under-use the window on purpose). #277 still fail-closes stale edits. buildModelMessages' persist/start rebuild still linear-shifts only when a blob is already over the 8 MiB store cap (one stringify on the in-cap product path). No live durable-turn round-trip in this attack.

Merge guidance

PASS WITH NOTES: safe to merge from this attack. Three prior CONCERNS Majors are on the head. Do not --approve from this skill.

What was not attacked

Live Workflows canvas, occupancy meter (#556), compaction (#552), Wasm status-bar paint of the window, tokenizer parity vs provider counts, models.dev limit.context schema drift beyond the parser tests, a planted oversize Blob that would still hit buildModelMessages' pre-existing linear byte-cap loop.