docs(platform): design the runnerhub per-router bounded send queue (RIG-1611) - #577
Open
rigel-mintaka wants to merge 1 commit into
Open
docs(platform): design the runnerhub per-router bounded send queue (RIG-1611)#577rigel-mintaka wants to merge 1 commit into
rigel-mintaka wants to merge 1 commit into
Conversation
rigel-mintaka
marked this pull request as ready for review
August 24, 2026 03:24
|
Compass engineering docs preview: https://compass-comms-rig-1611-runne.compass-eng-docs.pages.dev Deployed from Changed pages: |
…IG-1611) Close the delivery consumer's head-of-line blocking: the Server's single delivery-consumer goroutine funnels every deliver/steer/sweep/settle inline through the per-Runner `commandRouter`'s `sendMu` + flow-control-blocking `BidiStream.Send`, so one wedged Runner Sessions stream stalls delivery to every other session (including sessions on healthy Runners), settle/start draining, and bus-lag observation. The fix lives at the router, not in delivery: a bounded per-router outbound queue drained by one sender goroutine, with `send1`/`push`/`dispatch` becoming non-blocking enqueues. Overflow is per traffic class — deliver = synchronous refusal reusing the existing cursor/D2-sweep backstop, signal = best-effort drop, command = fail-fast (preserving OQ6 idempotent retry). The delivery consumer is untouched; no wire change; per-session ascending-seq order is preserved by the per-router FIFO. Server-side counterpart to the merged runner-side concurrent-dispatch work (SEA-1575); lands independently. Design record only — impl is filed as follow-up slices on merge-freeze. Ledger-impact: none Co-authored-by: Matt Wilkinson <matt@rigel.build>
rigel-mintaka
force-pushed
the
compass-comms/rig-1611-runnerhub-send-queue
branch
from
August 24, 2026 03:39
3f3d7bd to
2ab279f
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Close the delivery consumer's head-of-line blocking: the Server's single
delivery-consumer goroutine funnels every deliver/steer/sweep/settle inline
through the per-Runner
commandRouter'ssendMu+ flow-control-blockingBidiStream.Send, so one wedged Runner Sessions stream stalls delivery toevery other session (including sessions on healthy Runners), settle/start
draining, and bus-lag observation.
The fix lives at the router, not in delivery: a bounded per-router outbound
queue drained by one sender goroutine, with
send1/push/dispatchbecomingnon-blocking enqueues. Overflow is per traffic class — deliver = synchronous
refusal reusing the existing cursor/D2-sweep backstop, signal = best-effort
drop, command = fail-fast (preserving OQ6 idempotent retry). The delivery
consumer is untouched; no wire change; per-session ascending-seq order is
preserved by the per-router FIFO.
Server-side counterpart to the merged runner-side concurrent-dispatch work
(SEA-1575); lands independently. Design record only — impl is filed as
follow-up slices on merge-freeze.
Ledger-impact: none
Co-authored-by: Matt Wilkinson matt@rigel.build