Feature: support stream_id multiplexing for concurrent responses over a single WebSocket connection

[jtechapps]

Summary

The WebSocket Responses endpoint accepts a stream_id field on response.create, but it does not actually support multiplexing multiple concurrent responses over a single connection. This is a feature request to make stream_id a functional multiplexing key so that several in-flight responses (e.g. forked conversations) can share one WebSocket connection and be demultiplexed by the client.

Motivation

Clients that fork a conversation (one parent turn, multiple child turns chaining off the same previous_response_id) currently must open a separate WebSocket connection per concurrent child, because there is no reliable way to tell which events belong to which in-flight response on a shared socket. A working stream_id multiplexing scheme would let a client run N concurrent responses on a single connection, reducing connection overhead and matching the intent of the stream_id field the API already accepts.

Current behavior (observed)

Testing against the vLLM agentic-api WebSocket Responses endpoint (/v1/responses, model /models/models/GLM-5.2-FP8), with one parent turn (store: true) followed by two forks fired concurrently on the same connection — each with a distinct stream_id (forkA, forkB) and each chaining previous_response_id=<parent>:

1. stream_id is not echoed on any event. Across 123 streamed events, every event had stream_id: null. The value sent on response.create is accepted but never returned, so it cannot be used to route events.
2. Only wrapper events carry a response id. response.created / response.in_progress / response.completed embed the full response object (with id). The high-frequency streaming events — response.reasoning_text.delta, response.output_text.delta, response.output_item.added, etc. — carry only item_id, output_index, content_index, and a per-response sequence_number. They carry no response.id and no stream_id, so a delta cannot be attributed to a response.
3. Concurrent requests are serialized, not interleaved. The two forks did not interleave: forkA ran to completion (sequence_number 0→50, response.completed) before forkB began (its sequence_number reset to 0). Timestamps confirm no overlap (forkA created_at≈…494 → completed ≈…503; forkB started ≈…503 → completed ≈…517).

Note: fork inheritance itself works — both children correctly recalled the parent context (pineapple, zebra). The gap is specifically in multiplexing.

Why this blocks multiplexing

For a client to multiplex concurrent responses on one socket it needs a demux key present on every event. Today:

• stream_id is dropped, and
• delta events carry neither stream_id nor response.id.

So if the server ever did interleave two responses, their delta tokens would be un-attributable. Combined with the server serializing requests, there is currently no throughput or demux benefit to sending multiple response.create messages on one connection.

Requested behavior

1. Echo the client-supplied stream_id on every streamed event for that response (all response.* events, including every *.delta), so clients can demultiplex.
2. Allow multiple response.create requests to be genuinely in-flight concurrently on a single WebSocket connection, with their events interleaved and each tagged by stream_id.
3. (Alternative/if stream_id is not adopted) Include the owning response.id on every event as a demux key.

Environment

• Endpoint: vLLM agentic-api WebSocket Responses endpoint (/v1/responses), reached via kubectl port-forward svc/agentic-api -n agentic-api 19000:9000 → ws://localhost:19000/v1/responses
• Model: /models/models/GLM-5.2-FP8

Reproduction

Each response.create envelope:

{
  "type": "response.create",
  "model": "/models/models/GLM-5.2-FP8",
  "input": [{"role": "user", "content": "<text>"}],
  "max_output_tokens": 512,
  "stream_id": "<forkA|forkB>"
}

1. Open one WebSocket connection.
2. Send a parent response.create with "store": true, input "Remember these two facts: the fruit is pineapple and the animal is zebra. Acknowledge with OK.". Read to response.completed; capture response.id.
3. On the same connection, send two response.create messages back-to-back without waiting:
   • stream_id:"forkA", previous_response_id:<parent>, input "What fruit did I ask you to remember? Reply with only that word."
   • stream_id:"forkB", previous_response_id:<parent>, input "What animal did I ask you to remember? Reply with only that word."
4. Read all events on the single socket and attempt to route each event to forkA/forkB by its stream_id.

Observed: every event has stream_id: null (cannot route); delta events carry no response.id either; the two responses arrive serialized (forkA fully completes before forkB starts).
Expected (feature): every event carries the originating stream_id; the two responses may interleave; the client can demultiplex both concurrent streams over the one connection.