Summary
BATCH_OPERATOR_GROUPS is the first attestation buildenv drops when an outbound envelope overflows, and the copy it drops is deferred to the next envelope. Under the next-epoch roster lookahead (#595) that is precisely the envelope the roster was meant to authorise, so the outpost keeps a stale window, refuses envelope N's deliverer, and the roster that would repair it is sitting inside the envelope that was refused.
Mechanism
Three facts compose:
- BOG is queued last, so it holds the highest attestation id.
sysio.epoch.cpp queues ATTESTATION_TYPE_OPERATORS at :807 and ATTESTATION_TYPE_BATCH_OPERATOR_GROUPS at :934.
buildenv packs in id order and trims from the end. sysio.msgch.cpp:1967 — entries.pop_back().- Trimmed rows stay
READY and ride the next envelope. sysio.msgch.cpp:1972-73.
So the highest-id attestation is the first casualty of overflow, and that is always the roster.
Why it is reachable
sysio.epoch.hpp:164-171 already documents the size profile:
"at the ceiling the roster attestation alone is ~20 KB, about 62 % of a 32 KiB envelope (it was ~31 % of 64 KiB) … a roster that large would leave little room for value-bearing attestations in the same envelope, and buildenv would carry the remainder to the next epoch."
The platform envelope cap moved 65 536 → 32 768, so the headroom that made this theoretical is materially tighter. Any epoch with a large roster plus a normal load of value-bearing attestations can overflow.
Why it matters more now
Before the lookahead, a deferred roster meant the outpost's window was one envelope stale — recoverable, because the operator admitted under the old window could still deliver. With active_group_index naming the next epoch's group, the roster in envelope N is what authorises envelope N+1's deliverer. Dropping it is self-perpetuating rather than self-healing.
Found while reviewing #595; it predates that PR (the queue order and the trim are both pre-existing), so it is filed separately rather than fixed there.
Options
- Queue BOG before
OPERATORS so it takes the lowest id and packs first.
Cheap — an ordering change. But it reprioritises every envelope, permanently demoting OPERATORS and whatever else shares the budget. It also only moves the problem: whatever is now last becomes the new first casualty.
- Reserve bytes for BOG in the estimator/trim budget, so it is never a trim candidate.
Most faithful to intent — it says "the roster is structural, not discretionary". Needs a budget chosen against the value-bearing attestations it displaces, and a decision about what happens when the roster alone exceeds the reservation.
- Make BOG non-trimmable outright (skip it in the trim loop).
Simplest to state, but with no size bound it can starve an envelope entirely — at the 1000-member ceiling the roster is ~62 % of the cap on its own.
- Split the roster across envelopes / ship deltas.
Removes the size pressure at the source, but changes the wire contract, and batch-operator-schedule-window.md deliberately requires the whole window on every envelope so a late-joining or soft-reset outpost re-converges in one step.
Leaning (2), with (1) as a stopgap if a budget needs more thought than the fix warrants. Either way it wants its own tests: an overflow case asserting the roster survives, and one asserting whatever is displaced instead is genuinely deferrable.
Related
Summary
BATCH_OPERATOR_GROUPSis the first attestationbuildenvdrops when an outbound envelope overflows, and the copy it drops is deferred to the next envelope. Under the next-epoch roster lookahead (#595) that is precisely the envelope the roster was meant to authorise, so the outpost keeps a stale window, refuses envelope N's deliverer, and the roster that would repair it is sitting inside the envelope that was refused.Mechanism
Three facts compose:
sysio.epoch.cppqueuesATTESTATION_TYPE_OPERATORSat:807andATTESTATION_TYPE_BATCH_OPERATOR_GROUPSat:934.buildenvpacks in id order and trims from the end.sysio.msgch.cpp:1967—entries.pop_back().READYand ride the next envelope.sysio.msgch.cpp:1972-73.So the highest-id attestation is the first casualty of overflow, and that is always the roster.
Why it is reachable
sysio.epoch.hpp:164-171already documents the size profile:The platform envelope cap moved 65 536 → 32 768, so the headroom that made this theoretical is materially tighter. Any epoch with a large roster plus a normal load of value-bearing attestations can overflow.
Why it matters more now
Before the lookahead, a deferred roster meant the outpost's window was one envelope stale — recoverable, because the operator admitted under the old window could still deliver. With
active_group_indexnaming the next epoch's group, the roster in envelope N is what authorises envelope N+1's deliverer. Dropping it is self-perpetuating rather than self-healing.Found while reviewing #595; it predates that PR (the queue order and the trim are both pre-existing), so it is filed separately rather than fixed there.
Options
OPERATORSso it takes the lowest id and packs first.Cheap — an ordering change. But it reprioritises every envelope, permanently demoting
OPERATORSand whatever else shares the budget. It also only moves the problem: whatever is now last becomes the new first casualty.Most faithful to intent — it says "the roster is structural, not discretionary". Needs a budget chosen against the value-bearing attestations it displaces, and a decision about what happens when the roster alone exceeds the reservation.
Simplest to state, but with no size bound it can starve an envelope entirely — at the 1000-member ceiling the roster is ~62 % of the cap on its own.
Removes the size pressure at the source, but changes the wire contract, and
batch-operator-schedule-window.mddeliberately requires the whole window on every envelope so a late-joining or soft-reset outpost re-converges in one step.Leaning (2), with (1) as a stopgap if a budget needs more thought than the fix warrants. Either way it wants its own tests: an overflow case asserting the roster survives, and one asserting whatever is displaced instead is genuinely deferrable.
Related