feat(opp-solana): drive resumable inbound dispatch; delete the depot-side terminal budget

[valthon]

Summary

The depot-side half of resumable OPP dispatch (paired with wire-solana#419), plus a proto-generator fix that closes a decoder-panic wedge.

Previously the depot delivered an inbound envelope to the Solana outpost and settled it in one terminal epoch_in call, bounded by a fixed terminal budget the depot computed and enforced so the whole envelope fit one transaction. That cap made large envelopes undeliverable and coupled the depot to the outpost's per-transaction limits.

Commits

1. Drive resumable Solana dispatch from the depot — delete the depot-side terminal budget (solana_terminal_budget.hpp and its sysio.msgch enforcement) and drive the outpost's two-phase crank instead: sysio.msgch stages chunks and tips consensus, and the outpost_solana_client plugin settles via the permissionless dispatch_attestations instruction, reading the on-chain cursor and resuming until every attestation has dispatched. The relay's drain is ordered consensus-first (a cheap no-op before any per-attestation manifest work when consensus has not tipped), carries the remaining deadline budget into the drain so the dispatch-phase guard is reachable, and memoizes the delivered envelope only after the buffer is fully staged so a partial upload re-uploads from zero rather than cranking an incomplete buffer. The stub IDL tracks the outpost's slimmed seven-account epoch_in surface and the now read-only dispatch chunk_buffer; docs/opp-two-phase-dispatch-design.md records the design.

2. Guard nested decodes in the Solana proto generator — the generated decoders sliced a nested/length-delimited field without checking the length, so a malformed length prefix panicked instead of returning DecodeError — on-chain, a permanently wedged outpost epoch reachable by any depot that emits a structurally malformed field. Every length-delimited decode the generator emits (nested-message, repeated, and map templates and the emitted decode_bytes / skip_field runtime helpers) is now guarded with checked_add(...).ok_or(BufferOverflow)? before the bounds check. The checked add also closes an overflow route a bare bounds check misses: new_pos + len overflows usize before any comparison for a u64::MAX-scale varint length, and the program builds release with overflow-checks on. Regenerated decoders return DecodeError, and the outpost handlers' existing log-and-skip discipline does the rest. Runtime unit tests cover both routes.

Testing

• sysio.msgch contract tests (21 cases) and the outpost_solana_client plugin suite green after rebasing onto current master.
• Proto-generator jest + Rust runtime suites green (both decode routes under overflow-checks).
• End-to-end paired with wire-solana#419: the decoder guard was confirmed live by the wire-solana panic pins flipping before their retirement.

Addresses the review feedback on the prior revision (consensus-first drain ordering, remaining-budget deadline, inert-seed removal, stub-IDL account surface); the session-scaffolding docs flagged in review have been removed.

[heifner]

Review of the resumable-dispatch change: 10 findings, ranked by severity. Nine are anchored inline on the diff; one sits in a file this PR does not touch and is included here.

---

Retry idempotence / fee burn — plugins/batch_operator_plugin/src/outpost_opp_job.cpp:120 (file not in this diff)

Nothing records that chunks were already staged, and _last_outbound_epoch only advances on a clean return (outpost_opp_job.cpp:120-134). Every throw path in the new, much longer delivery (deadline, program abort, decode failure, stale cursor read) leaves it unset, so the cron re-enters deliver_outbound_envelope every 15 s, which unconditionally re-sends all data chunks from offset 0 (outpost_solana_client.cpp:908-923) before reaching the terminal loop. For a wedged epoch that is an unbounded fee burn and RPC load — 4 tx/tick for the dev-026 envelope, 99 tx/tick at the 64 KiB cap — with no backoff, and the re-upload itself eats into the same 15 s budget, making the deadline-throw wedge self-reinforcing. Suggest probing the chunk_buffer PDA (or a local memo) before re-uploading.

[valthon]

Rebased, addressed all comments, and new flow tests 13/13 ✅

[heifner]

Re-review at b349fe1414 (rebased onto dade24c613).

This is not a patch-up of the version I reviewed — it is a different architecture. epoch_in is now delivery-only (the dispatch_limit argument is gone entirely) and settlement moved to a new permissionless dispatch_attestations instruction driven from the on-chain cursor. That restructuring is what closes the prior findings, and it closes them at the root rather than at the symptom. All ten are addressed.

---

Prior findings

#	Finding	Status
1	Retry re-uploads all chunks on every drain failure	Resolved by design. The terminal epoch_in is fixed-size with no manifest, and the inline drain is try/caught (outpost_solana_client.cpp:1095-1107) so settlement can no longer fail the delivery. Re-upload now requires an actual chunk / terminal-call failure — the same surface as the base.
2	settled never seeded from the cursor → multi-round wedge	Fixed. drive_dispatch_rounds reads the cursor before packing (:597, :612). Pinned by drive_dispatch_resumes_from_nonzero_cursor.
3	Zero-attestation / decode-failure path never records the delivery	Fixed, and better than asked. Delivery records regardless of decode; decode failure now throws inside drain_dispatch where it belongs (:993-1001); an empty envelope gets one clamped close crank, since the program's completion block is the only place next_epoch_index advances. The matching program-side regression test (dispatch_window(0, 0, 5) == (0, 0)) is the right place for that invariant to live.
4	Cursor read at confirmed against a write confirmed at processed	Fixed. The read is pinned to commitment_t::processed (:748).
5	Silent round exhaustion; budget not sized against the depot's worst case	Fixed. Exhaustion elogs (:689-692), the budget went 64 → 128 with a stated sizing rationale, and resumption is now real via drain-then-read.
6	MAX_TERMINAL_DYNAMIC_ACCOUNTS = 16 left unmeasured after the SEC-94 fixture was deleted	Fixed. dispatch_attestations_full_manifest_fits_packet_limit measures the real shape — IDL statics, fee payer, the heap-frame pre-instruction, and 16 distinct writable extras.
7	Conflicts with the standing outpost-remit-is-immediate rule	Addressed; one item outstanding — see below.
8	queueout lost its only chain_code registration check	Fixed. The check is back with its original message, and buildenv / dispatch_attestation now carry explicit assert messages instead of the bare key not found.
9	The dispatch state machine shipped untested; five dead extractor views	Fixed. The loop is factored over its RPC seams with 12 dedicated tests, and the dead views are deleted.
10	Lockstep break against the manifest-pinned program	Fixed. wire-solana#419 now carries MAX_CHUNK_BYTES = 668 and dispatch_attestations.

The e2e evidence checks out independently: run 31550041513 succeeded with BRANCH_WIRE_SYSIO and BRANCH_WIRE_SOLANA both set to feat/resumable-opp-dispatch and was dispatched after the head commit — the correct paired combination for a coupled change.

---

New findings

Four on the reworked design, anchored inline. Two of them (the memoize placement and the eager manifest build) are worth fixing before merge; the other two are cleanups.

---

Please remove docs/superpowers/plans/2026-08-11-contained-resumable-dispatch-relay.md

That file is session scaffolding rather than repository documentation: a checkbox task list addressed "For agentic workers", pinning an absolute local path (/home/valthon/wire-platform/wire-sysio) and a commit-by-commit workflow that is already history. It will read as normative to the next person who greps docs/ and finds it.

docs/opp-two-phase-dispatch-design.md is the genuine design doc and should absolutely stay — it is the artifact that makes this change reviewable, and its outpost-remit-is-immediate section is doing real work. docs/superpowers/specs/2026-08-11-contained-resumable-dispatch-relay-design.md is a placement redirect for the plan; if the plan goes, it is worth asking whether the redirect still has a reader.

---

Outstanding landing item

wire-platform-manifest/.claude/rules/outpost-remit-is-immediate.md is still unedited, and continues to state: "There is no concept of 'pending remit,' 'queued release,' … the dispatch loop IS the release."

The design doc handles the tension honestly — there is an explicit "Relationship to outpost-remit-is-immediate" section that concedes the design fails the rule's letter and argues its intent, with the rule update listed as landing step 4 — and wire-solana#419 has already updated opp-outpost-technical-spec.md (+106/−57). Only the manifest rule is left. Worth landing in the same window, since every diagnosis playbook built on that rule will otherwise misread production.

---

Verdict: the architecture change is the right call, and the tests now pin the exact region where both confirmed bugs lived. Nothing here blocks the coordinated merge with wire-solana#419.

[heifner]

The memo is set before the chunks are staged, and a partially-staged buffer cannot be cranked — the comment asserts the opposite.

The comment reads "Memoize BEFORE the sends: even a partially-staged delivery leaves this relay able to crank once consensus tips via other operators." The program refuses exactly that case: read_staged_envelope (wire-solana instructions/opp/mod.rs:381-384) requires parsed.received_chunks == parsed.total_chunks, else TerminalChunkBeforeDataComplete; a buffer that was never created fails the owner check one step earlier.

So when the chunk loop throws partway (RPC hiccup, deadline, validator hiccup), _delivered_envelope is already set and the delivery propagates out to outpost_opp_job::run_outbound's catch with _last_outbound_epoch unset. Every read_inbound_envelope tick until the next successful outbound then takes the drain-then-read path at :1130, builds the manifest, and — once consensus tips via other operators — sends dispatch_attestations against our own incomplete buffer. skip_preflight is false (libfc solana_client.cpp:1726), so it is rejected at simulation rather than landing: no fee, but a guaranteed-to-fail RPC round trip plus a wlog on every inbound tick in that window.

Bounded — the next run_outbound tick re-uploads from offset zero and repairs it — but the window is real and the comment points the next reader the wrong way.

Suggest moving the assignment below the chunk loop (before or after the terminal epoch_in). The goal the comment actually wants — crank when our terminal call or drain failed but the buffer is complete — is served either way, and the partial case it claims to cover is not reachable by design.

[heifner]

The cost claim here is wrong: the drain builds the full effect manifest before it ever checks consensus or the cursor.

"Consensus-not-reached is a cheap internal no-op (one progress read) inside the drive loop" is true of drive_dispatch_rounds — but drain_dispatch does all of its manifest work first. extract_inbound_effects walks the envelope, then accounts_for_effect runs per effect, and every reserve-backed shape reaches reserve_info_for_codes, which costs two get_account_info calls (the Reserve PDA, then config_pda; the per-(token, reserve) cache does not span calls, and the config read sits inside the cached function rather than beside it).

For every operator except the one whose delivery tips consensus, that entire manifest is built and discarded — on every inbound tick, for as long as the epoch's outbound envelope is missing, which is precisely the window this call is designed to run in.

Scoped honestly: a steady-state envelope carrying only OPERATORS / BATCH_OPERATOR_GROUPS produces no reserve-backed effects and costs nothing extra. This bites on swap and reserve epochs — the busy ones, where the same envelope also has the most effects to walk.

Reading progress once at the top of drain_dispatch and returning early on !consensus_reached || dispatched_count >= total_attestations fixes both the cost and the comment, and subsumes the redundant pre-loop read flagged at :597.

[heifner]

The pre-loop cursor read and this guard are both dead — one get_account_info per drain with no behavioural contribution.

settled from :597 is consumed only by this line. Three lines down, :605 re-reads the same account and :612 overwrites settled before any other use, so the value from :597 never reaches the packing loop.

And this guard cannot fire for round ≥ 1 either: the bottom of the loop already returns on settled >= total_attestations (:680), so re-entry guarantees settled < total_attestations.

Initialising settled = 0 and dropping both lines is behaviour-identical. Walked against all twelve drive_dispatch_* expectations:

• already_drained_cursor_sends_nothing — falls through to :605, consensus is reached, :612 sets settled = 3, :631 returns without sending. Same outcome.
• consensus_not_reached_sends_nothing — reaches the :606 check one read earlier and returns. Same outcome.
• resumes_from_nonzero_cursor — unaffected; :612 is what seeds the resume, which is the fix for the original finding.

The only observable delta is that an already-drained epoch whose consensus flag is false now logs "consensus not yet reached" instead of returning silently, which is if anything the more accurate line. Worth doing because this drain now runs on every inbound tick, so the saved read is per-operator per-tick rather than once per envelope.

[heifner]

This hands drain_dispatch the original deadline duration rather than the remaining budget — the read path gets it right, this one does not.

read_inbound_envelope computes deadline_abs - fc::time_point::now() (:1132-1133); here the full deadline is passed after the chunk uploads have already consumed part of it, so drain_dispatch installs a fresh budget at :862.

Wall-clock is still capped, so this is not an overrun: fc::task::deadline_scope keeps the earliest of the previous and new deadlines (libfc fc/task/deadline.hpp:31-36), and http_client consults fc::task::current_deadline() (http_client.cpp:147), so RPCs still throw on the caller's real deadline.

The cost is diagnostic. The loop's own throw_if_past_deadline(deadline_abs, OP_DISPATCH_ATTESTATIONS) at :1021 can never fire first, so the OP_DISPATCH_ATTESTATIONS label — added by this PR specifically to identify this phase — is unreachable, and an operator whose drain runs long sees a generic transport timeout instead. Matching the read path's deadline_abs - fc::time_point::now() restores it.

[heifner]

Please drop this file from the PR.

It is session scaffolding rather than repository documentation — a checkbox task list addressed "For agentic workers", pinning an absolute local path (/home/valthon/wire-platform/wire-sysio), a build directory, and a commit-by-commit workflow that is already history. Its constraints are also written against a 114bfbd486 baseline that this branch has since been rebased off, so anyone who finds it under docs/ will be reading instructions that no longer describe the tree.

docs/opp-two-phase-dispatch-design.md is the genuine design doc and should stay — it is what makes this change reviewable. If this plan goes, docs/superpowers/specs/2026-08-11-contained-resumable-dispatch-relay-design.md is worth a second look too, since it exists to redirect this plan's placement.

[heifner]

Approving. All fourteen findings from the previous rounds are addressed, and the two-phase split is the right resolution rather than a patch — the instruction reads its own cursor and dispatch_limit bounds only how far, which makes the unseeded-cursor defect unrepresentable. drive_dispatch_rounds factored over its RPC touchpoints turns the previously untestable region into the best-covered part of the file, and dispatch_attestations_full_manifest_fits_packet_limit makes MAX_TERMINAL_DYNAMIC_ACCOUNTS a measured budget again.

Non-blocking follow-ups:

Generator templates ship untested. The new commit's payload is four length-delimited templates in field.ts; the tests added cover only the emitted runtime helpers, which were already bounds-checked (just the checked_add route is new there). Field.test.ts touches nothing that emits a decode arm, and no fixture proto declares a map<, so genMapDecode — two of the four edits — has no coverage at all, jest or cargo. A map field in sample_types.proto plus a truncated-prefix and a u64::MAX-prefix case in the fixture crate would close it.

The guard reaches no deployed program yet. wire-solana pins wire-opp-solana-models at exactly 1.0.38; only the platform e2e overrides it with the generated tree, which is why the panic-pin evidence is real. The published crate still has the unguarded decoders, so the landing sequence needs a republish and pin bump alongside wire-solana#419.

outpost-remit-is-immediate is unreconciled. The companion updated opp-outpost-technical-spec.md, but the standing rule in wire-platform-manifest still says there is no queued release and that the dispatch loop is the release. The design doc names the file; it needs to land in the same sequence.

Three envelope parses per drain, one above the consensus gate. The decode probe, count_inbound_attestations and extract_inbound_effects each parse up to 64 KiB, on every inbound tick while the outbound envelope is missing. The probe's placement is deliberate, but consensus-not-tipped means the same bytes fail identically next tick, so every non-tipping operator pays a parse for a result that cannot matter yet. One parse can feed both the count and the effects.

The already-drained early return did not land. drain_dispatch reads progress at the gate but uses only consensus_reached, so a drained epoch still walks every effect and does two account reads per unique reserve before drive_dispatch_rounds returns on its cursor check. Composes with the above — the count plus the progress already read gives the early return for free.

Minor. SWAP_REMIT_ATTESTATION_TYPE and SOL_OUTPOST_ID are now unused in sysio.msgch_tests.cpp, with a stray double blank line where emitted_attestation_count was. drive_dispatch_rounds does two progress reads per round where the next round's could serve the previous round's stall check. The .gitignore commit says in its own message that it is unrelated to the branch, which argues for its own PR.

[heifner]

Cross-PR note, not a change request on what is already approved here.

#555 (WIRE-295) rewrites the same three outpost files from the pre-two-phase design and conflicts three-ways with this branch on all of them. Most of it is superseded: its fail-closed parse_terminal_manifest_envelope patches the five whole-envelope extractors this PR deletes, and the intent survives in drain_dispatch's decode probe; its underwriter commit is already on master, so that half is an empty delta. Two parts are not superseded and would be lost if #555 is closed as obsolete.

Pinned Reserve custody. #555 replaces resolve_token_custody(config, token_code) with resolve_reserve_custody(reserve), reading custody_mint and custody_decimals off the Reserve account already in hand. That matches the program: both wire-solana#403 and wire-solana#419 branch on reserve.custody_mint / reserve.custody_decimals and never consult token_addresses_by_code for this path. As it stands here, re-pointing a token's config row after a Reserve is created makes the relay derive the destination ATA for the new mint while the handler derives it from the pinned one — require_remaining_account then returns EffectAccountMissing and aborts. Under the cursor design that holds dispatched_count rather than failing a single delivery, and the greedy packer always retakes the attestation at the cursor, so it does not converge on retry. It also removes the second get_account_info(config_pda) per reserve.

This is the same defect class as the open collateral-custody finding on #553 — custody read from the mutable config while the program branches on a pinned mint — so adopting the pinned-facts pattern here sets it for both halves.

SWAP_REMIT SPL surface. #555 adds the custody mint, the recipient wallet, the ATA program and the system program to the remit manifest — the surface swap_revert already carries here, and swap_remit does not.

resolve_token_custody_reads_outpost_config_maps and resolve_token_custody_requires_both_config_entries pin the current behaviour and would need to move with it.

[heifner]

Approving at 6db0483ee.

Re-reviewed the two commits added since the last round — d1159239 (harden the relay against epoch churn) and 6db0483e (gate the stub IDL fixture against the generated liqsol_core IDL) — plus the rebase onto current master. The dispatch state machine reads correctly: no defect in drive_dispatch_rounds packing or cursor resume, in the degrade/throw split in reserve_info_for_codes, or in the new boot asserts. On the generator commit, the checked_add guards plus the end > data.len() bounds check now present on all four templates close the slice-panic route.

Two things I raised in re-review and am withdrawing:

• sysio.msgch.wasm grew 155,599 → 184,678 B on a net-deleting source commit. Different CDT options produce different wasm sizes; the larger artifact is intended. No action.
• Boot asserts hard-fail against the currently deployed program. has_idl("dispatch_attestations") and the dispatched_count shape check both fail against wire-solana@next, but fail-loud is the right design here and the merge order (wire-solana#419 first, then this) is understood.

Nothing else blocks. The items below are follow-ups, not change requests.

dispatch_limit is bounded by accounts, never by compute

The greedy packer's only exit is the account union exceeding MAX_TERMINAL_DYNAMIC_ACCOUNTS:

if (candidate.size() > MAX_TERMINAL_DYNAMIC_ACCOUNTS && batch_end > settled) break;

Only four attestation types produce an inbound_effect — OPERATOR_ACTION (deposit/withdraw), DEPOSIT_REVERT, SWAP_REMIT, SWAP_REVERT. Everything else contributes zero accounts, so candidate.size() never grows and the window extends to the end of the envelope. send_dispatch_attestations requests a heap frame but deliberately no set_compute_unit_limit, so the transaction runs under the runtime default. Nor does packet size bound it: a window of N no-account attestations serializes to the same small instruction regardless of N.

In practice this is hard to reach — the zero-account types are state mirrors (OPERATORS, BATCH_OPERATOR_GROUPS) that ride one per envelope, and the effect-yielding types bound the window normally. The realistic shape is an epoch dominated by OperatorAction(SLASH), which falls in the no-effect arm. Worth a ceiling anyway, because the failure mode has no self-heal: on CU exhaustion send_dispatch throws, the cursor does not advance, and every later tick repacks the byte-identical window from the same cursor. A constant beside MAX_TERMINAL_DYNAMIC_ACCOUNTS, or a set_compute_unit_limit, closes it. This one had an implicit bound from the depot-side terminal budget that this PR removes, so it is worth not leaving open indefinitely.

solana_idl_stub_consistency_test fails rather than skips on a stale sibling

The script's skip path covers an absent target/idl/liqsol_core.json, but a present but older one is a hard exit 1. Against wire-solana@next (5eac77cb) there are three divergences: no dispatch_attestations instruction, no EpochDeliveries.dispatched_count, and epoch_in declares 12 accounts against the stub's 7. This resolves itself once #419 is the manifest default, and CI without a built sibling skips — so the only cost is a false red for anyone holding a stale build. Gating the comparison on the sibling declaring dispatch_attestations (skip-with-reason otherwise) would remove that window.

Related: tests/CMakeLists.txt:290 always passes --solana-path ${CMAKE_SOURCE_DIR}/../wire-solana, and resolve_solana_path gives the argument precedence, so the documented WIRE_SOLANA_PATH override never fires under ctest. In any workspace where wire-solana is not the literal sibling the gate takes the skip path and is permanently green — the silent no-op the docstring's precedence rule exists to prevent.

Carried over, still open

• drain_dispatch has no already-drained early return. It reads progress only to gate on consensus_reached, discards dispatched_count, then builds the full manifest — one get_account_info per distinct reserve — before drive_dispatch_rounds returns immediately on its cursor check. Re-entered on every inbound tick while the delivered envelope matches the epoch.
• Three full parses of the same envelope per drain — the decode probe, extract_inbound_effects, and count_inbound_attestations. One decode can feed all three.
• Two cursor reads per dispatch round. The post-send read_progress() is overwritten by the next round's top-of-round read with the same value; only the consensus re-check needs it, and consensus cannot un-tip.
• Dead constants in sysio.msgch_tests.cpp — SOL_OUTPOST_ID and SWAP_REMIT_ATTESTATION_TYPE lost their last users when the two SVM budget tests were deleted. UNCOVERED_TEST_ATTESTATION_TYPE and emitted_attestation_count were removed correctly; these two were missed. Unused internal-linkage constexpr trips -Wunused-const-variable under -Wall.

Landing checklist

wire-platform-manifest/.claude/rules/outpost-remit-is-immediate.md is still unedited across three rounds now. It states remits fire "inline, in the same tx that processes the envelope" and explicitly bans a two-step shape; this PR makes settlement a separate permissionless instruction. docs/opp-two-phase-dispatch-design.md argues the invariant is preserved in spirit and names the rule update as a landing step, and the companion already updated opp-outpost-technical-spec.md — only the manifest rule is left. It lives in another repo, so it is the item most likely to be dropped once this merges, and every stall-diagnosis playbook is built on the sentence it contradicts.

[valthon]

https://github.com/Wire-Network/wire-platform-build-system/actions/runs/31815724193