inference: cache-stable two-block system prompt (don’t bust the KV prefix)

[btipling]

Parent

#548 (agent session architecture)

Problem

OpenCode (and similar) pin a two-part system array so provider KV cache
stays valid across turns: if the first block is unchanged, later turns hit
cache instead of re-prefilling the world.

We already see cache working (1.2M cached on the usage chip) — the
prefix is enormous. Session-varying bits (cwd, attached skill bodies,
envelope-derived addenda, working notes once #550 exists) mixed into one
blob will invalidate that prefix whenever they twitch.

New-session “thinking hang” is still mostly serial I/O + cold start, but a
stable prefix is how we keep later turns from paying 1.2M uncached
again.

Goal

Assemble system as two (or few) stable blocks:

1. Stable: tool schemas + persona snapshot + invariant harness law.
2. Session-varying: cwd, skill catalog (skills: inject a catalog, not full bodies, every turn #557), working notes (agent memory: durable working notes / findings across turns (identity, not one-shot) #550),
   turn-specific addenda.

Do not mutate block 1 mid-session unless the operator actually changed
persona or the tool surface.

Success: consecutive turns on the same model show a high cache-read /
input ratio on the occupancy/totals chip without stuffing new skill
bodies into block 1.

Constraints

• Gateway / AI SDK must actually accept a multi-part system (or a single
  string with a documented stable prefix). Lock in create-plan against
  live runAgent — do not invent cache_control if the provider options
  we send cannot express it.
• Authoring meta-tools on every turn (meta(#531): built-in meta persona + skill authoring tools on /api/agent #533/meta(#532): built-in meta sandbox tools on /api/agent #535) still enlarge block 1.
  Shrinking that surface is related but can be a later issue; this one
  is ordering / stability, not “delete meta tools.”
• Not status bar: show current context occupancy in addition to summed turn usage #556 (how we paint occupancy).

Maps from peer harnesses

OpenCode system.ts + concatenate-into-two-blocks for Anthropic cache;
plugin transform only if it does not smash the first element.

Suggested next

create-plan with #557 (catalog vs bodies) — same assembly site.

Orrery (2026-08-15)

Choice precedes assembly. Resolve model (operator catalog / Next) then tools, system blocks, history. No generic prompt with the id slapped on.

Layout, strictly: system → tool definitions → durable spec / notes (#550) → plan/todo → history (#549). Nothing above history may vary turn-to-turn for a given (session, model) except at a phase/compact boundary.

Stay-put is a routing decision. Next mid-session is a family switch: #551/#552 treat it as a cache event, not a free paint change.

Breakpoints (if Gateway/provider can express them) sit at the stable/volatile cut. Do not invent cache_control the live runAgent path cannot send.

[btipling]

2026-08-19 source-level peer research

Two-block system is the right product. Name the peer mechanisms so the create-plan does not reinvent them:

Context Epoch / world-state diffs (steal the invariant)

Peer	Guidance freshness without busting prefix cache
Codex	WorldState sections snapshot() + render_diff(previous). Unchanged AGENTS.md / tools / permissions → no new tokens. Replacement notice when they do change.
OpenCode V2	Context Epoch: freeze concatenated SystemContext as an immutable baseline; later AGENTS.md / skill / MCP changes are chronological system messages; baseline is rewritten only at compaction (or incompatible transition).
DeepSeek	Re-read / rebase AGENTS.md at agent/pre-step after compact shadows the old instruction event.
OMP	AGENTS.md loaded once at session start (like our persona snapshot). Sticky RULES.md re-attached near the leaf so compact cannot bury it; TTSR injections survive compact.

Our persona snapshot (#485) is already an OpenCode baseline for standing orders. #558 generalizes that:

1. Block 1 (baseline): tool schemas + persona snapshot + harness law. Byte-stable for a (session, model) until compact / persona-New / tool-surface change.
2. Block 2 (volatile): cwd, skill catalog (skills: inject a catalog, not full bodies, every turn #557), agent memory: durable working notes / findings across turns (identity, not one-shot) #550 notes if they churn, agent memory: tool_result snapshots are not live file views (re-read or they’re stale) #693 re-read reminder, turn addenda.
3. Mid-session skill/MCP/tool-list change → a diff system message (Codex render_diff), not a rewrite of block 1. Honor it at the same phase boundaries as compaction: model-capability context compaction (not API payload slices) #552, not mid-tool-streak (already in the Orrery note).
4. Compaction is the moment the prefix is allowed to change (new epoch).

OMP sticky-rules lesson: if we will not rebase personas mid-session (we will not — snapshot lock), re-attach the isolated persona/rules block near the current turn so A4 cannot bury standing orders. That is #670’s salience, durable across compact — not a live-body rewrite.

Handbook inject (#694)

If/when the later handbook ships, memory_summary goes in block 2 on New session, never block 1, and never mid-turn. OMP learn waits until the next session on purpose.

Skip: inventing cache_control the live runAgent path cannot send (already in the body).