feat: multiplex response streams over WebSocket

[franciscojavierarceo]

Summary

• add typed stream_id routing to the Responses WebSocket API and echo it on every event, including local and execution errors
• run distinct stream lanes concurrently while preserving FIFO execution within the same stream and the backward-compatible default lane
• bound outstanding requests, aggregate input, event queues, upstream responses, gateway outputs, and MCP discovery/materialization so multiplexing cannot grow memory without limit
• document the WebSocket contract and add unit/integration coverage for interleaving, ordering, capacity, shutdown, cancellation, and resource limits

Test Plan

• cargo test --workspace (all runnable tests pass; PostgreSQL-only tests remain ignored without TEST_POSTGRES_URL)
• cargo build --workspace --bins
• cargo clippy --workspace --all-targets --all-features -- -D warnings
• cargo fmt --all -- --check
• git diff --check
• uvx pre-commit run --all-files
• gstack review, Rust-specific review, adversarial review, and final testing/performance review report no remaining findings

Closes #239

[Copilot]

Warning

Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.

Pull request overview

This PR adds stream_id-based multiplexing to the Responses WebSocket API, enabling concurrent response streams per connection with FIFO ordering per stream, while also enforcing request/event/tool/MCP size bounds to prevent unbounded memory growth.

Changes:

• Added stream_id routing for WebSocket response.create and echoed it on all related events (including error events), with per-lane concurrency + FIFO ordering.
• Introduced and enforced multiple capacity limits (outstanding WebSocket requests, aggregate input bytes, upstream SSE line size, normalized event size, shared response budget across MCP discovery + upstream + gateway output).
• Added/updated docs and tests covering ordering, interleaving, capacity, cancellation, shutdown draining, and bounded outputs.

Reviewed changes

Copilot reviewed 24 out of 25 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
docs/api/index.md	Documents stream_id multiplexing and new capacity/budget limits.
crates/agentic-server/tests/responses_websocket_test.rs	Adds integration coverage for lane concurrency/FIFO, capacity, shutdown, and cancellation.
crates/agentic-server/tests/oidc_auth_test.rs	Updates OIDC WebSocket expiry test to assert stream_id is echoed on auth errors.
crates/agentic-server/src/handler/websocket/responses.rs	Implements multiplexed WebSocket scheduling, per-lane FIFO, and outbound buffering with stream_id tagging.
crates/agentic-server/src/handler/websocket/error.rs	Adds a 429 TooManyRequests WebSocket error and refines which errors can produce frames.
crates/agentic-server-core/src/tool/web_search.rs	Bounds web_search provider responses and aggregate output; bounds You.com response reads while streaming.
crates/agentic-server-core/src/tool/registry.rs	Adds MCP declaration/tool/byte limits; introduces guarded tool registry building with shared budgeting.
crates/agentic-server-core/src/tool/mcp/mod.rs	Wires in bounded HTTP transport module for MCP.
crates/agentic-server-core/src/tool/mcp/handler.rs	Bounds MCP tool output size while composing/serializing results.
crates/agentic-server-core/src/tool/mcp/client.rs	Switches MCP HTTP transport to bounded client implementation.
crates/agentic-server-core/src/tool/mcp/bounded_http.rs	New bounded MCP HTTP/SSE client to reject oversized messages/events early.
crates/agentic-server-core/src/tool/handler.rs	Defines shared MAX_GATEWAY_TOOL_OUTPUT_BYTES constant (1 MiB).
crates/agentic-server-core/src/executor/upstream.rs	Enforces shared response budget across streamed lines and blocking bodies; uses bounded event sender.
crates/agentic-server-core/src/executor/response_budget.rs	New shared per-request response budget primitive (1 MiB).
crates/agentic-server-core/src/executor/mod.rs	Registers new response_budget module.
crates/agentic-server-core/src/executor/inference.rs	Bounds upstream response body reads and SSE line length while decoding.
crates/agentic-server-core/src/executor/gateway_accumulator.rs	Adds bounded stream event channel + per-event size cap; supports explicit sequence for error events.
crates/agentic-server-core/src/executor/gateway.rs	Enforces tool output sizing, integrates response-budget consumption, and adds shared materialization semaphore.
crates/agentic-server-core/src/executor/engine.rs	Plumbs shared response budget through MCP discovery, upstream rounds, and gateway execution; uses bounded stream event channel.
crates/agentic-server-core/src/executor/compaction.rs	Updates call site for fetch_blocking_payload new signature.
crates/agentic-server-core/Cargo.toml	Adds sse-stream dependency for bounded SSE parsing.
Cargo.toml	Adds workspace dependency on sse-stream.
CHANGELOG.md	Notes multiplexing + new bounding behavior in changelog.
ARCHITECTURE.md	Updates architecture docs for multiplexing, FIFO lanes, and new bounding/backpressure model.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[LOGO127]

AI-assisted local verification on 6f1b1da found a small test-fixture flake in oversized_terminal_response_is_not_persisted (CPU, WSL, Rust 1.98.0, locked dependencies).

With only diagnostic context added to the assertion, the sixth focused run failed before any response.created; the collected event was a 502 LLM unavailable, rather than the intended oversized-event error. This reproduces without the proposed continuation/session changes.

streaming_execution_context returns its static SSE response without consuming the upload. Hyper 1.10.1's HTTP/1 dispatcher can close the read side when an unread body is dropped, so the large instructions upload can fail before reaching the executor's terminal-event check. A minimal fixture adjustment is to consume the body before returning SSE:

axum::routing::post(|_body: axum::body::Bytes| async {
    ([(axum::http::header::CONTENT_TYPE, "text/event-stream")], UPSTREAM_SSE)
})

I also checked the fixture directly with a gated two-part body via Handler::call: it must remain pending after the first part, then produce the same SSE after the remainder is released. That deterministic test fails before the body extractor and passes afterwards; the original persistence test then passed 30/30 focused repetitions. Its event-size and no-persistence assertions are unchanged.

The isolated one-file test-only candidate passes the full Rust workspace: 908 passed, 0 failed, 9 ignored (8 PostgreSQL cases and 1 doctest). Workspace/all-target Clippy with warnings denied, rustfmt, and applicable hooks also pass. This is local verification, not upstream CI or a real-model test. No production path was changed and no competing PR was opened. Happy to provide the focused regression patch if useful; this does not attempt to take over the multiplexing implementation.

[maralbahari]

@franciscojavierarceo I dont have further comments for this PR. can we merge #186 first since it has been through rounds or review and resolving conflict? then on this PR you resolve conflict?