diff --git a/docs-site/src/content/docs/ja/reference/configuration/server.md b/docs-site/src/content/docs/ja/reference/configuration/server.md index 9760a04017..3ae0fa70ab 100644 --- a/docs-site/src/content/docs/ja/reference/configuration/server.md +++ b/docs-site/src/content/docs/ja/reference/configuration/server.md @@ -15,7 +15,7 @@ description: リスナー、リモート アクセス、アドミッション | `stallTimeoutSec?` | `number` | `300` | `response.incomplete` より前にアップストリーム データがない秒数。最小 1。 | `connectTimeoutMs?` | `number` | `200000` |試行ごとの DNS/TCP/TLS/最終ヘッダーの期限。本体が生成される前に終了します。 | | `shutdownTimeoutMs?` | `number` | `5000` |アクティブなターンが中止される前の正常な排出期限。 | -| `websockets?` | `boolean` | `false` |応答 WebSocket パスとして `supports_websockets` をアドバタイズします。 False は HTTP/SSE を維持します。 | +| `websockets?` | `boolean` | `false` | クライアント向け Responses WebSocket パスを広告して許可します。false の場合クライアントは HTTP/SSE を使いますが、対象となる canonical ChatGPT upstream WS 最適化は無効にしません。 | | `corsAllowOrigins?` | `string[]` | `[]` | 追加の正確な CORS origin。ループバック origin は常に許可します。`chrome-extension://` など authority ベースのブラウザー拡張 origin に対応し、`*` はワイルドカードではありません。Firefox と Safari は拡張 UUID を(インストール/ブラウザー起動ごとに)再生成するため、origin が変わったらエントリを更新してください。 | | `apiKeys?` | `OcxApiKey[]` | `[]` |生成された `ocx_…` 資格情報は、非ループバック バインドでの管理およびデータ プレーン認証によって受け入れられました。ダッシュボードで管理。 | | `storageCleanupPolicy?` | `StorageCleanupPolicy` |無効 |アーカイブされたセッションのクリーンアップ ポリシーをオプトインします。暗黙的に有効になることはありません。 | diff --git a/docs-site/src/content/docs/ko/reference/architecture.md b/docs-site/src/content/docs/ko/reference/architecture.md index 0166f48292..1554597a66 100644 --- a/docs-site/src/content/docs/ko/reference/architecture.md +++ b/docs-site/src/content/docs/ko/reference/architecture.md @@ -117,6 +117,14 @@ Responses 항목 타입으로 구분됩니다 — 따라서 MCP 네임스페이 반환하고, Codex는 해당 세션에서 HTTP로 폴백합니다. `"websockets": true`가 설정되면 같은 엔드포인트가 업그레이드를 받아들이고 WebSocket 브리지를 사용합니다. +이 클라이언트 설정과 별개로, 루트 `stream: true`인 canonical ChatGPT forward 요청은 +stable Bun 1.4.0 이상에서 Codex 업스트림 WebSocket을 사용할 수 있습니다. 번들 Bun 1.3.14, +prerelease, 또는 검증할 수 없는 런타임 identity는 HTTP/SSE를 사용합니다. 성공한 업스트림 WS +응답은 같은 downstream SSE 계약을 유지하며, 원시 JSON WebSocket 프레임과 downstream SSE +envelope를 각각 4 MiB로 제한하고 8 MiB producer queue 상한이 있는 bounded eager single-reader +relay를 거칩니다. queue overflow 시 업스트림을 닫고 downstream에는 +terminal `response.failed` 이벤트와 `[DONE]`을 내보냅니다. + Codex 컨텍스트 compaction은 라우팅된 모델에서도 동작합니다. `server/responses/compact.ts`는 `POST /v1/responses/compact`를 내부 라우팅 요약 턴으로 처리해 압축된 히스토리를 반환합니다. `responses/parser.ts`와 `bridge.ts`는 remote compaction v2의 `compaction_trigger` 턴을 처리해 diff --git a/docs-site/src/content/docs/ko/reference/configuration/server.md b/docs-site/src/content/docs/ko/reference/configuration/server.md index ad12aead09..8a0626fc85 100644 --- a/docs-site/src/content/docs/ko/reference/configuration/server.md +++ b/docs-site/src/content/docs/ko/reference/configuration/server.md @@ -15,7 +15,7 @@ description: 리스너, 원격 접근, admission 키, 타임아웃, 저장소, | `stallTimeoutSec?` | `number` | `300` | 업스트림 데이터가 없을 때 `response.incomplete`가 되기까지의 초 수입니다. 최소 1입니다. | | `connectTimeoutMs?` | `number` | `200000` | 시도별 DNS/TCP/TLS/최종 헤더 기한입니다. 본문 생성 전에 끝납니다. | | `shutdownTimeoutMs?` | `number` | `5000` | 진행 중인 turn을 중단하기 전에 허용하는 정상 종료 드레인 기한입니다. | -| `websockets?` | `boolean` | `false` | Responses WebSocket 경로에 `supports_websockets`를 광고합니다. `false`이면 HTTP/SSE를 유지합니다. | +| `websockets?` | `boolean` | `false` | 클라이언트용 Responses WebSocket 경로를 광고하고 허용합니다. `false`이면 클라이언트는 HTTP/SSE를 사용하며, 적격 canonical ChatGPT 업스트림 WS 최적화는 비활성화하지 않습니다. | | `corsAllowOrigins?` | `string[]` | `[]` | CORS에서 추가로 허용할 정확한 origin입니다. 루프백 origin은 항상 허용됩니다. `chrome-extension://` 같은 authority 기반 브라우저 확장 origin을 지원하며, `*`는 와일드카드가 아닙니다. Firefox와 Safari는 확장 UUID를 (설치/브라우저 실행 때마다) 새로 만드므로 origin이 바뀌면 항목을 갱신하세요. | | `apiKeys?` | `OcxApiKey[]` | `[]` | 비루프백 바인드에서 관리 API와 데이터 플레인 인증이 허용하는 생성된 `ocx_…` 자격 증명입니다. 대시보드에서 관리합니다. | | `storageCleanupPolicy?` | `StorageCleanupPolicy` | disabled | 선택적으로 활성화하는 보관 세션 정리 정책입니다. 절대 암묵적으로 활성화되지 않습니다. | diff --git a/docs-site/src/content/docs/ko/reference/proxy-formats.md b/docs-site/src/content/docs/ko/reference/proxy-formats.md index 39497eef4b..532fe0b4ef 100644 --- a/docs-site/src/content/docs/ko/reference/proxy-formats.md +++ b/docs-site/src/content/docs/ko/reference/proxy-formats.md @@ -65,6 +65,12 @@ deltas, 그리고 정확히 하나의 종료 `response.completed`, `response.fai 클라이언트로 전달되는 Responses SSE 프레임은 SSE 블록 구분자 앞의 원시 바이트 기준으로 프레임당 4 MiB로 제한됩니다. HTTP에서는 구분자 없이 이 한도를 초과한 업스트림 프레임을 합성 `response.failed` 이벤트와 이어지는 `data: [DONE]`으로 fail closed 처리합니다. Responses WebSocket 브리지에서는 같은 조건에서 502 `websocket_protocol_error`를 보내고 업스트림 reader를 취소합니다. 완전한 Responses 종료 프레임이 이미 수신된 경우에는 그 종료가 우선하며, 이후의 과도한 크기 또는 잘못된 바이트는 완료된 턴을 전송 오류로 바꾸지 않고 버립니다. +canonical ChatGPT forward streaming은 stable Bun 1.4.0 이상에서 Codex 업스트림 WebSocket을 +투명하게 사용할 수 있습니다. 번들 Bun 1.3.14, prerelease, 또는 검증 불가능한 런타임 identity는 +HTTP/SSE를 사용합니다. 업스트림 WS adapter는 같은 downstream SSE 계약을 유지하며, 원시 JSON +프레임과 SSE envelope를 각각 4 MiB로 제한하고 8 MiB byte queue가 넘치기 전에 업스트림을 닫습니다. +queue overflow 시 downstream에는 terminal `response.failed` 이벤트와 `[DONE]`을 내보냅니다. + 모든 종료 Responses usage 객체에는 제공자가 해당 세부 정보를 보고하지 않았더라도 두 상세 객체가 모두 포함됩니다. @@ -88,6 +94,9 @@ deltas, 그리고 정확히 하나의 종료 `response.completed`, `response.fai 있습니다. 인증과 origin admission은 WebSocket 핸드셰이크 동안 처리됩니다. 각 프레임 안에서 다시 반복되지는 않습니다. +이 클라이언트 업그레이드는 위의 투명한 업스트림 ChatGPT WebSocket 선택과 별개이며, +`websockets` 설정은 클라이언트 엔드포인트만 제어합니다. + 클라이언트는 JSON 텍스트 프레임을 보냅니다. ```json diff --git a/docs-site/src/content/docs/reference/architecture.md b/docs-site/src/content/docs/reference/architecture.md index d47e140dca..afb65c00a9 100644 --- a/docs-site/src/content/docs/reference/architecture.md +++ b/docs-site/src/content/docs/reference/architecture.md @@ -141,6 +141,13 @@ WebSocket upgrade while `websockets` is `false`, opencodex returns `426 upgrade_ falls back to HTTP for that session. When `"websockets": true` is set, the same endpoint accepts the upgrade and uses the WebSocket bridge. +Independently of that client-facing setting, canonical ChatGPT forward requests with root-level +`stream: true` may use Codex's upstream WebSocket transport on stable Bun 1.4.0 or newer. +Bundled Bun 1.3.14, prereleases, and unverifiable runtime identities use HTTP/SSE. Successful +upstream WS responses keep the downstream SSE contract and bypass `tee()` through a bounded eager +single-reader relay (4 MiB per raw/enveloped frame and an 8 MiB producer queue). Queue overflow +closes the upstream and emits a terminal downstream `response.failed` event followed by `[DONE]`. + Codex context compaction works for routed models. `server/responses/compact.ts` handles `POST /v1/responses/compact` by running an internal routed summarization turn and returning compacted history, while `responses/parser.ts` and `bridge.ts` handle remote compaction v2 diff --git a/docs-site/src/content/docs/reference/configuration/server.md b/docs-site/src/content/docs/reference/configuration/server.md index 88fd05fce4..81e6b3651b 100644 --- a/docs-site/src/content/docs/reference/configuration/server.md +++ b/docs-site/src/content/docs/reference/configuration/server.md @@ -16,7 +16,7 @@ runs helper features around provider requests. | `stallTimeoutSec?` | `number` | `300` | Seconds without upstream data before `response.incomplete`. Minimum 1. | | `connectTimeoutMs?` | `number` | `200000` | Per-attempt DNS/TCP/TLS/final-header deadline; it ends before body generation. | | `shutdownTimeoutMs?` | `number` | `5000` | Graceful drain deadline before active turns are aborted. | -| `websockets?` | `boolean` | `false` | Advertise `supports_websockets` for the Responses WebSocket path. False keeps HTTP/SSE. | +| `websockets?` | `boolean` | `false` | Advertise and admit the client-facing Responses WebSocket path. False keeps clients on HTTP/SSE; it does not disable an eligible canonical ChatGPT upstream WS optimization. | | `corsAllowOrigins?` | `string[]` | `[]` | Additional exact origins allowed by CORS. Loopback origins are always allowed. Authority-based browser extension origins such as `chrome-extension://` are supported; `*` is not a wildcard. Firefox and Safari regenerate the extension UUID (per install / per browser launch), so update the entry when the origin changes. | | `apiKeys?` | `OcxApiKey[]` | `[]` | Generated `ocx_…` credentials accepted by management and data-plane auth on non-loopback binds. Dashboard-managed. | | `storageCleanupPolicy?` | `StorageCleanupPolicy` | disabled | Opt-in archived-session cleanup policy. Never enabled implicitly. | diff --git a/docs-site/src/content/docs/reference/proxy-formats.md b/docs-site/src/content/docs/reference/proxy-formats.md index bd02453019..32391e5b42 100644 --- a/docs-site/src/content/docs/reference/proxy-formats.md +++ b/docs-site/src/content/docs/reference/proxy-formats.md @@ -72,6 +72,13 @@ bridge, the same condition emits a 502 `websocket_protocol_error` and cancels th A complete Responses terminal frame is authoritative: oversized or malformed trailing bytes after that terminal are dropped rather than replacing the completed turn with a transport failure. +For canonical ChatGPT forward streaming, stable Bun 1.4.0 or newer may transparently use +Codex's upstream WebSocket transport. Bundled Bun 1.3.14, prereleases, and unverifiable runtime +identities use HTTP/SSE. The upstream WS adapter keeps the same downstream SSE contract, caps both +the raw JSON frame and its SSE envelope at 4 MiB, and closes the upstream when its 8 MiB byte queue +would overflow. That overflow emits a terminal downstream `response.failed` event followed by +`[DONE]`. + Every terminal Responses usage object includes both detail objects, even when the provider did not report those details: @@ -95,6 +102,9 @@ When `websockets` is enabled, a client may upgrade `/v1/responses` instead of op Authentication and origin admission happen during the WebSocket handshake. They are not repeated inside each frame. +This client-facing upgrade is separate from the transparent upstream ChatGPT WebSocket selection +described above; the `websockets` setting controls only the client-facing endpoint. + The client sends JSON text frames: ```json diff --git a/docs-site/src/content/docs/ru/reference/configuration/server.md b/docs-site/src/content/docs/ru/reference/configuration/server.md index b799c56549..df0ff57401 100644 --- a/docs-site/src/content/docs/ru/reference/configuration/server.md +++ b/docs-site/src/content/docs/ru/reference/configuration/server.md @@ -16,7 +16,7 @@ description: Listener, удалённый доступ, admission key, тайм | `stallTimeoutSec?` | `number` | `300` | Секунды без upstream-данных до `response.incomplete`. Минимум 1. | | `connectTimeoutMs?` | `number` | `200000` | Дедлайн одной попытки DNS/TCP/TLS/final-header; он завершается до генерации тела ответа. | | `shutdownTimeoutMs?` | `number` | `5000` | Дедлайн graceful-drain до принудительного прерывания активных turn'ов. | -| `websockets?` | `boolean` | `false` | Объявлять `supports_websockets` для WebSocket-пути Responses. Значение false удерживает HTTP/SSE. | +| `websockets?` | `boolean` | `false` | Объявляет и разрешает клиентский WebSocket-путь Responses. При false клиенты используют HTTP/SSE; это не отключает подходящую upstream WS-оптимизацию canonical ChatGPT. | | `corsAllowOrigins?` | `string[]` | `[]` | Дополнительные точные origin, разрешённые CORS. Loopback-origin разрешены всегда. Поддерживаются authority-based origin браузерных расширений, например `chrome-extension://`; `*` не является маской. Firefox и Safari пересоздают UUID расширения (при каждой установке/запуске браузера), поэтому обновляйте запись при смене origin. | | `apiKeys?` | `OcxApiKey[]` | `[]` | Сгенерированные credentials `ocx_…`, принимаемые для management и data-plane auth на не-loopback bind'ах. Управляются через дашборд. | | `storageCleanupPolicy?` | `StorageCleanupPolicy` | disabled | Opt-in policy очистки архивированных сессий. Никогда не включается неявно. | diff --git a/docs-site/src/content/docs/zh-cn/reference/configuration/server.md b/docs-site/src/content/docs/zh-cn/reference/configuration/server.md index ecc3ce1bf4..3030232964 100644 --- a/docs-site/src/content/docs/zh-cn/reference/configuration/server.md +++ b/docs-site/src/content/docs/zh-cn/reference/configuration/server.md @@ -16,7 +16,7 @@ description: 监听、远程访问、准入密钥、超时、存储、侧车、 | `stallTimeoutSec?` | `number` | `300` | 在上游没有数据之前可等待的秒数,超过后返回 `response.incomplete`。最小值为 1。 | | `connectTimeoutMs?` | `number` | `200000` | 每次尝试的 DNS/TCP/TLS/最终响应头截止时间;它在正文生成之前结束。 | | `shutdownTimeoutMs?` | `number` | `5000` | 优雅停机截止时间,超过后会中止仍在进行中的请求。 | -| `websockets?` | `boolean` | `false` | 为 Responses WebSocket 路径声明 `supports_websockets`。设为 false 会保留 HTTP/SSE。 | +| `websockets?` | `boolean` | `false` | 声明并允许面向客户端的 Responses WebSocket 路径。设为 false 时客户端使用 HTTP/SSE;它不会禁用符合条件的 canonical ChatGPT 上游 WS 优化。 | | `corsAllowOrigins?` | `string[]` | `[]` | CORS 额外允许的精确 origin。loopback origin 始终允许;支持 `chrome-extension://<扩展 ID>` 等基于 authority 的浏览器扩展 origin,`*` 不是通配符。Firefox 和 Safari 会(每次安装/启动浏览器时)重新生成扩展 UUID,origin 变化后请更新该条目。 | | `apiKeys?` | `OcxApiKey[]` | `[]` | 管理平面和非回环绑定上的数据平面身份验证可接受的已生成 `ocx_…` 凭据。由仪表板管理。 | | `storageCleanupPolicy?` | `StorageCleanupPolicy` | disabled | 可选启用的归档会话清理策略。不会被隐式启用。 | diff --git a/docs-site/src/content/docs/zh-tw/reference/configuration/server.md b/docs-site/src/content/docs/zh-tw/reference/configuration/server.md index 68b6ef516d..4d9f5921d1 100644 --- a/docs-site/src/content/docs/zh-tw/reference/configuration/server.md +++ b/docs-site/src/content/docs/zh-tw/reference/configuration/server.md @@ -15,7 +15,7 @@ description: 監聽器、遠端存取、許可金鑰、逾時、儲存、sidecar | `stallTimeoutSec?` | `number` | `300` | 在 `response.incomplete` 前無上游資料的秒數。最小 1。 | | `connectTimeoutMs?` | `number` | `200000` | 每次嘗試的 DNS/TCP/TLS/final-header 截止時間;它在 body 生成前結束。 | | `shutdownTimeoutMs?` | `number` | `5000` | 在中止活躍回合前的優雅排空截止時間。 | -| `websockets?` | `boolean` | `false` | 為 Responses WebSocket 路徑廣告 `supports_websockets`。False 保持 HTTP/SSE。 | +| `websockets?` | `boolean` | `false` | 廣告並允許面向 client 的 Responses WebSocket 路徑。False 時 client 使用 HTTP/SSE;不會停用符合條件的 canonical ChatGPT upstream WS 最佳化。 | | `corsAllowOrigins?` | `string[]` | `[]` | 額外的精確 CORS 來源。回送來源恆被允許。 | | `apiKeys?` | `OcxApiKey[]` | `[]` | 生成的 `ocx_…` 憑證,在非回送綁定上被管理與 data-plane 認證接受。由儀表板管理。 | | `storageCleanupPolicy?` | `StorageCleanupPolicy` | 停用 | 選擇加入的已封存 session 清理政策。永不隱含啟用。 | diff --git a/src/server/index.ts b/src/server/index.ts index 099991d272..c9328287c2 100644 --- a/src/server/index.ts +++ b/src/server/index.ts @@ -417,6 +417,8 @@ function attachLiveSidebandUpstream( // upstream cannot hold Codex open after response.completed; darwin no-rewrite traffic // requires explicit config-eager opt-in (`auto` always stays tee on darwin). // selectEagerPath(process.platform, needsClientRewrite, config.streamMode ?? "auto") +// Codex upstream WS runtime gating and the forced bounded single-reader branch +// are owned by responses/ws-upstream.ts and responses/core.ts respectively. // relaySseEagerBounded(upstreamResponse.body, turnAc, // new Response(eagerBody, // Default shape (tee + background inspection): diff --git a/src/server/responses/core.ts b/src/server/responses/core.ts index 67b30c86e4..a8886f8725 100644 --- a/src/server/responses/core.ts +++ b/src/server/responses/core.ts @@ -190,6 +190,7 @@ import { } from "../responses-terminal-repair"; import { isWin32EagerRewrite, selectEagerPath } from "../../lib/bun-stream-caps"; import { cancelBodyOnAbort } from "../../lib/abort"; +import { isCodexWsUpstreamResponse, type BunRuntimeGateInput } from "./ws-upstream"; import { createResponsesItemIdPayloadRewrite, hasResponsesItemIdRepair, @@ -418,6 +419,7 @@ interface CodexPoolAccountRetryArgs { // needs the inbound scope or the retry could land on a different wire than the // first attempt. inboundWire?: InboundWire; + codexWsRuntimeIdentity?: BunRuntimeGateInput; translatorBudget: TranslatorBudget; turnAdmissionLease?: AdmissionLease; }; @@ -590,7 +592,7 @@ async function retryCodexPoolOnAlternateAccount( upstream.signal, connectMs, stream, - providerFetch(route.provider), + providerFetch(route.provider, options.codexWsRuntimeIdentity), // Credential-bearing forward send: never follow a redirect into a // dead-host rejection after the credential was seen (#914). route.provider.authMode === "forward", @@ -730,6 +732,8 @@ export interface HandleResponsesOptions { onNativePassthroughCancel?: () => void; /** Internal deterministic clock/timer seam for provider terminal repair. */ responsesTerminalRepairScheduler?: ResponsesTerminalRepairScheduler; + /** Internal deterministic runtime-identity seam for Codex upstream WS selection tests. */ + codexWsRuntimeIdentity?: BunRuntimeGateInput; /** * When true, body `prompt_cache_key` is a Claude Desktop shared cache cohort * (system/tools hash), not a per-session id — do not use it for Anthropic pool affinity. @@ -2266,7 +2270,8 @@ async function handleResponsesInner( method: request.method, headers: request.headers, body: request.body, - }, recovery), upstream.signal, connectMs, parsed.stream, providerFetch(route.provider), + }, recovery), upstream.signal, connectMs, parsed.stream, + providerFetch(route.provider, options.codexWsRuntimeIdentity), route.provider.authMode === "forward") // Every real attempt response — including an intermediate 5xx the // retry wrapper replaces — proves the host was reached (#914 review). @@ -2327,7 +2332,8 @@ async function handleResponsesInner( method: request.method, headers: request.headers, body: request.body, - }, recovery), upstream.signal, connectMs, parsed.stream, providerFetch(route.provider), + }, recovery), upstream.signal, connectMs, parsed.stream, + providerFetch(route.provider, options.codexWsRuntimeIdentity), route.provider.authMode === "forward") .then(res => { settleObservedHostResponse(); @@ -2574,9 +2580,14 @@ async function handleResponsesInner( needsClientRewrite, config.streamMode ?? "auto", ); + // A successful Codex WS upgrade is a push source. If it entered tee(), + // the inspection branch could drain continuously while the slow client + // branch retained bytes without a bound. Force the existing bounded, + // single-reader relay before tee; HTTP fallback responses stay unmarked. + const forceCodexWsEagerRelay = isCodexWsUpstreamResponse(upstreamResponse); const inlineEagerRewrite = needsClientRewrite - && (win32EagerRewrite || eagerPath?.useEagerRelay === true); - if (eagerPath?.useEagerRelay || win32EagerRewrite) { + && (forceCodexWsEagerRelay || win32EagerRewrite || eagerPath?.useEagerRelay === true); + if (forceCodexWsEagerRelay || eagerPath?.useEagerRelay || win32EagerRewrite) { const turnAc = new AbortController(); linkAbortSignal(upstream, turnAc.signal); registerTurn(turnAc, options.turnAdmissionLease); @@ -2634,9 +2645,9 @@ async function handleResponsesInner( onDone: () => unregisterTurn(turnAc), }, inlineEagerRewrite ? { rewriteBudget: translatorBudget } : undefined); // When selected, this relay closes response.completed even if upstream - // keeps the connection alive. Windows forced-rewrite traffic and Darwin - // explicit eager traffic apply client rewrites inline rather than via - // the tee()+JS-pull chain. + // keeps the connection alive. Marked Codex WS traffic, Windows + // forced-rewrite traffic, and Darwin explicit eager traffic apply + // client rewrites inline rather than via the tee()+JS-pull chain. if (!headers.has("content-type")) headers.set("content-type", "text/event-stream"); return markEagerRelaySseResponse( markNativePassthroughSseResponse(new Response(eagerBody, { @@ -2926,7 +2937,7 @@ async function handleResponsesInner( : clampImageMaxRounds(config.images?.videoMaxRounds ?? 2), connectTimeoutMs: config.connectTimeoutMs ?? 200_000, stallTimeoutSec: config.stallTimeoutSec, - fetchImpl: providerFetch(route.provider), + fetchImpl: providerFetch(route.provider, options.codexWsRuntimeIdentity), onRequestBuilt: request => recordAdapterReasoning(logCtx, request), ...(vidPlan?.timeoutMs ? { videoTimeoutMs: vidPlan.timeoutMs } : {}), onUsage: usage => { @@ -3250,7 +3261,8 @@ async function handleResponsesInner( method: builtInitialRequest.method, headers: builtInitialRequest.headers, body: builtInitialRequest.body, - }, recovery), upstream.signal, connectMs, parsed.stream, providerFetch(route.provider)); + }, recovery), upstream.signal, connectMs, parsed.stream, + providerFetch(route.provider, options.codexWsRuntimeIdentity)); }, { abortSignal: upstream.signal, label: safeHostLabel(builtInitialRequest.url) }, ); @@ -3329,7 +3341,8 @@ async function handleResponsesInner( ? await activeAdapter.fetchResponse(retryRequest, { abortSignal: upstream.signal, timeoutMs: connectMs, stream: parsed.stream }) : await fetchWithHeaderTimeout(retryRequest.url, { method: retryRequest.method, headers: retryRequest.headers, body: retryRequest.body, - }, upstream.signal, connectMs, parsed.stream, providerFetch(route.provider)); + }, upstream.signal, connectMs, parsed.stream, + providerFetch(route.provider, options.codexWsRuntimeIdentity)); } finally { retryRequest.releaseBodyObservation?.(); } @@ -3653,7 +3666,7 @@ async function handleResponsesInner( upstream.signal, connectMs, nextParsed.stream, - providerFetch(route.provider), + providerFetch(route.provider, options.codexWsRuntimeIdentity), ); }, { abortSignal: upstream.signal, label: safeHostLabel(builtContinuationRequest.url) }, diff --git a/src/server/responses/fetch-helpers.ts b/src/server/responses/fetch-helpers.ts index b4affdc0ab..f780c0aa0f 100644 --- a/src/server/responses/fetch-helpers.ts +++ b/src/server/responses/fetch-helpers.ts @@ -1,5 +1,10 @@ import type { Server } from "bun"; -import { codexWsUpstreamFetch, shouldUseCodexWsUpstream } from "./ws-upstream"; +import { + codexWsUpstreamFetch, + currentBunRuntimeIdentity, + shouldUseCodexWsUpstream, + type BunRuntimeGateInput, +} from "./ws-upstream"; import { bridgeToResponsesSSE, buildResponseJSON, formatErrorResponse, type ResponsesTerminalStatus } from "../../bridge"; import { getConfigPath, @@ -131,14 +136,17 @@ export function safeOriginLabel(url: string): string { -export function providerFetch(provider: OcxProviderConfig): typeof globalThis.fetch { +export function providerFetch( + provider: OcxProviderConfig, + runtime: BunRuntimeGateInput = currentBunRuntimeIdentity(), +): typeof globalThis.fetch { const base = (provider as OcxProviderConfig & { fetch?: typeof globalThis.fetch }).fetch ?? globalThis.fetch; // ChatGPT Codex backend: streaming turns ride the responses_websockets // transport (measured ~3s faster TTFT than the SSE POST queue); everything // else keeps the provider's HTTP fetch. See ws-upstream.ts for the details. const wrapped = (input: Parameters[0], init?: RequestInit) => { - if (typeof input === "string" && init && shouldUseCodexWsUpstream(input, init)) { - return codexWsUpstreamFetch(input, init, base); + if (typeof input === "string" && init && shouldUseCodexWsUpstream(input, init, runtime)) { + return codexWsUpstreamFetch(input, init, base, runtime); } return base(input, init); }; diff --git a/src/server/responses/ws-upstream.ts b/src/server/responses/ws-upstream.ts index 5f0092ac48..d17236d98c 100644 --- a/src/server/responses/ws-upstream.ts +++ b/src/server/responses/ws-upstream.ts @@ -12,14 +12,79 @@ // returned event frames as an SSE byte stream, so every downstream consumer // (passthrough relay, adapter parsers, usage sniffing) is unchanged. +import { MAX_CLIENT_SSE_FRAME_BYTES } from "../sse-frame-buffer"; +import { compareBunVersions } from "../../lib/bun-stream-caps"; + const CODEX_RESPONSES_HTTP_URL = "https://chatgpt.com/backend-api/codex/responses"; const CODEX_RESPONSES_WS_URL = "wss://chatgpt.com/backend-api/codex/responses"; const WS_BETA = "responses_websockets=2026-02-06"; // If the 101 never arrives (network black hole), give SSE a chance well before // the caller's connect timeout (default 200s) would fire. const UPGRADE_DEADLINE_MS = 10_000; +// Keep the push-based WS transport inside the same memory envelope as the +// bounded SSE relays that consume this response. Unlike fetch response bodies, +// a WebSocket cannot be paused when a ReadableStream applies backpressure, so +// an upstream that outruns the consumer must be disconnected. +export const MAX_CODEX_WS_FRAME_BYTES = MAX_CLIENT_SSE_FRAME_BYTES; +export const MAX_CODEX_WS_QUEUE_BYTES = 8 * 1024 * 1024; +export const MIN_BOUNDED_CODEX_WS_BUN_VERSION = "1.4.0"; + +export type BunRuntimeIdentity = { + version: string; + versionWithSha: string; +}; + +export type BunRuntimeGateInput = string | BunRuntimeIdentity; + +const codexWsUpstreamResponses = new WeakSet(); + +/** True only for a successful Codex WebSocket upgrade, never an HTTP fallback. */ +export function isCodexWsUpstreamResponse(response: Response): boolean { + return codexWsUpstreamResponses.has(response); +} + +export function currentBunRuntimeIdentity(): BunRuntimeIdentity { + return { + version: Bun.version, + versionWithSha: Bun.version_with_sha, + }; +} -export function shouldUseCodexWsUpstream(url: string, init?: RequestInit): boolean { +function boundedRelayVersion(input: BunRuntimeGateInput): string | null { + if (typeof input === "string") return input.trim() || null; + const numericVersion = input.version.trim(); + const numericMatch = /^(\d+\.\d+\.\d+)$/.exec(numericVersion); + const detailedMatch = /^v?(\d+\.\d+\.\d+(?:-[0-9A-Za-z.-]+)?)\s+\([0-9a-fA-F]+\)$/.exec( + input.versionWithSha.trim(), + ); + if (!numericMatch || !detailedMatch) return null; + const detailedNumeric = /^(\d+\.\d+\.\d+)/.exec(detailedMatch[1])?.[1]; + return detailedNumeric === numericMatch[1] ? detailedMatch[1] : null; +} + +/** + * Bun 1.3.14 does not propagate a stalled HTTP response socket back to a JS + * ReadableStream producer on Windows. A real raw-TCP slow-client probe drained + * the entire upstream despite the eager relay queue; Bun 1.4.0-canary.1 stopped + * below one MiB. Prereleases still fail closed; release builds before 1.4.0 + * fall back to HTTP SSE. + */ +export function bunSupportsBoundedCodexWsRelay( + runtime: BunRuntimeGateInput = currentBunRuntimeIdentity(), +): boolean { + const version = boundedRelayVersion(runtime); + if (!version) return false; + if (/^\d+\.\d+\.\d+-/.test(version.trim())) return false; + const comparison = compareBunVersions(version, MIN_BOUNDED_CODEX_WS_BUN_VERSION); + return comparison !== null && comparison >= 0; +} + +export function shouldUseCodexWsUpstream( + url: string, + init?: RequestInit, + runtime: BunRuntimeGateInput = currentBunRuntimeIdentity(), +): boolean { + if (!bunSupportsBoundedCodexWsRelay(runtime)) return false; if (url !== CODEX_RESPONSES_HTTP_URL) return false; if ((init?.method ?? "GET").toUpperCase() !== "POST") return false; const body = init?.body; @@ -41,7 +106,11 @@ export function codexWsUpstreamFetch( url: string, init: RequestInit, sseFallback: typeof globalThis.fetch, + runtime: BunRuntimeGateInput = currentBunRuntimeIdentity(), ): Promise { + if (!bunSupportsBoundedCodexWsRelay(runtime)) { + return sseFallback(url, init); + } const signal = init.signal ?? undefined; if (signal?.aborted) { return Promise.reject(signal.reason ?? new DOMException("The operation was aborted.", "AbortError")); @@ -91,6 +160,13 @@ export function codexWsUpstreamFetch( let controller: ReadableStreamDefaultController | null = null; const encoder = new TextEncoder(); + const failStream = (message: string) => { + if (terminal) return; + terminal = true; + try { controller?.error(new Error(message)); } catch { /* stream already done */ } + try { ws.close(); } catch { /* already closing */ } + }; + const upgradeTimer = setTimeout(() => { if (opened || settledPreOpen) return; settledPreOpen = true; @@ -110,12 +186,14 @@ export function codexWsUpstreamFetch( reject(signal?.reason ?? new DOMException("The operation was aborted.", "AbortError")); return; } - try { ws.close(); } catch { /* already closing */ } if (controller && !terminal) { terminal = true; // Mirror an aborted fetch: the body read rejects with the abort reason. try { controller.error(signal?.reason ?? new DOMException("The operation was aborted.", "AbortError")); } catch { /* stream already done */ } } + // Error the body before close(): test doubles and some runtimes dispatch + // close synchronously, and the caller's abort reason must stay authoritative. + try { ws.close(); } catch { /* already closing */ } }; signal?.addEventListener("abort", onAbort, { once: true }); @@ -138,19 +216,33 @@ export function codexWsUpstreamFetch( const stream = new ReadableStream({ start(c) { controller = c; }, cancel() { try { ws.close(); } catch { /* already closing */ } }, - }); - resolve(new Response(stream, { + }, new ByteLengthQueuingStrategy({ highWaterMark: MAX_CODEX_WS_QUEUE_BYTES })); + const response = new Response(stream, { status: 200, // The 101 response headers (x-codex-*-reset-at quota hints) are not // exposed by Bun's WebSocket; the periodic quota poller covers those. headers: { "content-type": "text/event-stream; charset=utf-8" }, - })); + }); + codexWsUpstreamResponses.add(response); + resolve(response); }); ws.addEventListener("message", (event) => { if (!controller || terminal) return; const text = typeof event.data === "string" ? event.data : ""; if (!text) return; + // UTF-8 byte length is always at least the JS string length. Reject this + // cheap lower bound before parsing so an obviously oversized frame does + // not create another large object graph. + if (text.length > MAX_CODEX_WS_FRAME_BYTES) { + failStream("codex websocket frame exceeds the response size limit"); + return; + } + const encodedText = encoder.encode(text); + if (encodedText.byteLength > MAX_CODEX_WS_FRAME_BYTES) { + failStream("codex websocket frame exceeds the response size limit"); + return; + } let type: unknown; try { type = (JSON.parse(text) as { type?: unknown }).type; } catch { return; } if (typeof type !== "string") return; @@ -158,9 +250,26 @@ export function codexWsUpstreamFetch( // frames (codex.rate_limits, responsesapi.websocket_timing) are dropped // so downstream clients see exactly the stream shape they always got. if (!type.startsWith("response.") && type !== "error") return; + const prefix = encoder.encode(`event: ${type}\ndata: `); + const suffix = encoder.encode("\n\n"); + const frameBytes = prefix.byteLength + encodedText.byteLength + suffix.byteLength; + if (frameBytes > MAX_CLIENT_SSE_FRAME_BYTES) { + failStream("codex websocket frame exceeds the response size limit"); + return; + } + const availableBytes = controller.desiredSize ?? 0; + if (frameBytes > availableBytes) { + failStream("codex websocket response exceeded the buffered queue limit"); + return; + } + const sseFrame = new Uint8Array(frameBytes); + sseFrame.set(prefix); + sseFrame.set(encodedText, prefix.byteLength); + sseFrame.set(suffix, prefix.byteLength + encodedText.byteLength); try { - controller.enqueue(encoder.encode(`event: ${type}\ndata: ${text}\n\n`)); + controller.enqueue(sseFrame); } catch { + failStream("codex websocket response stream closed while enqueueing"); return; } if (type === "response.completed" || type === "response.failed" || type === "response.incomplete" || type === "error") { diff --git a/structure/04_transports-and-sidecars.md b/structure/04_transports-and-sidecars.md index 48d6b20763..a776a2bca0 100644 --- a/structure/04_transports-and-sidecars.md +++ b/structure/04_transports-and-sidecars.md @@ -92,6 +92,17 @@ The two-shape contract is mirror-commented in `src/server/index.ts`; the real and the platform matrix lives in `tests/bun-stream-caps.test.ts`. Keep all three in lockstep with any passthrough-policy change. +Canonical ChatGPT forward streaming has one transport-specific exception. A +stable Bun runtime at or above 1.4.0 may use Codex's upstream +`responses_websockets` transport; bundled Bun 1.3.14, prereleases, and +unverifiable runtime identities stay on HTTP/SSE. A successful upstream WS +response is re-encoded to the same SSE surface and forced through the bounded +eager single-reader relay instead of `tee()`: raw and enveloped frames are capped +at 4 MiB and the WS producer queue at 8 MiB. Overflow closes the upstream and +the downstream relay emits its terminal `response.failed` event plus `[DONE]`. +Pre-open HTTP fallback remains unmarked and follows the ordinary configured +stream path. + Translated response request-log tracking and the heartbeat relay also reuse `createSseInspector`. This keeps every client-facing SSE observation path on the same byte-bounded, discard-and-resynchronize frame policy and ensures the @@ -243,6 +254,10 @@ HTTP/SSE. When true, Codex may use Responses WebSocket frames handled by `src/se If Codex still attempts a WebSocket upgrade while the feature is disabled, `/v1/responses` rejects the upgrade with 426 so Codex falls back to HTTP cleanly. +That setting controls the client-facing upgrade only. The transparent upstream +ChatGPT WS optimization described above is selected independently and still +returns the same downstream SSE contract. + The endpoint handles `response.create`, ignores `response.processed`, supports warmup `generate: false`, and feeds the same request pipeline as HTTP/SSE. diff --git a/tests/passthrough-abort.test.ts b/tests/passthrough-abort.test.ts index 2b98ce91a5..113bd27f0c 100644 --- a/tests/passthrough-abort.test.ts +++ b/tests/passthrough-abort.test.ts @@ -57,7 +57,8 @@ describe("passthrough relayWithAbort (RC2, passthrough path)", () => { expect(sseBranch).toContain("const needsClientRewrite = clientBlockRewrite !== undefined;"); expect(sseBranch).toContain("new Response(eagerBody"); expect(sseBranch).toContain("const rewrittenBody = clientBlockRewrite !== undefined"); - expect(sseBranch).toContain("eagerPath?.useEagerRelay || win32EagerRewrite"); + expect(sseBranch).toContain("isCodexWsUpstreamResponse(upstreamResponse)"); + expect(sseBranch).toContain("forceCodexWsEagerRelay || eagerPath?.useEagerRelay || win32EagerRewrite"); expect(sseBranch).not.toContain("win32TerminalRelay"); // #864: win32 traffic that DOES need a client rewrite takes the eager single // reader with the payload rewrite applied inline — never the tee()+JS-pull diff --git a/tests/ws-upstream.test.ts b/tests/ws-upstream.test.ts index b58472eabb..545fc57698 100644 --- a/tests/ws-upstream.test.ts +++ b/tests/ws-upstream.test.ts @@ -1,9 +1,33 @@ import { afterEach, describe, expect, jest, test } from "bun:test"; import { providerFetch } from "../src/server/responses/fetch-helpers"; -import { codexWsUpstreamFetch, shouldUseCodexWsUpstream } from "../src/server/responses/ws-upstream"; +import { handleResponses } from "../src/server/responses"; +import { isEagerRelaySseResponse } from "../src/server/relay"; +import { + bunSupportsBoundedCodexWsRelay, + codexWsUpstreamFetch as rawCodexWsUpstreamFetch, + currentBunRuntimeIdentity, + isCodexWsUpstreamResponse, + MAX_CODEX_WS_FRAME_BYTES, + MAX_CODEX_WS_QUEUE_BYTES, + shouldUseCodexWsUpstream as rawShouldUseCodexWsUpstream, +} from "../src/server/responses/ws-upstream"; import type { OcxProviderConfig } from "../src/types"; +import type { OcxConfig } from "../src/types"; const CODEX_URL = "https://chatgpt.com/backend-api/codex/responses"; +const BOUNDED_WS_RUNTIME = "1.4.0"; + +function shouldUseCodexWsUpstream(url: string, init?: RequestInit): boolean { + return rawShouldUseCodexWsUpstream(url, init, BOUNDED_WS_RUNTIME); +} + +function codexWsUpstreamFetch( + url: string, + init: RequestInit, + fallback: typeof fetch, +): Promise { + return rawCodexWsUpstreamFetch(url, init, fallback, BOUNDED_WS_RUNTIME); +} function streamingInit(body: Record = {}): RequestInit { return { @@ -14,6 +38,57 @@ function streamingInit(body: Record = {}): RequestInit { } describe("shouldUseCodexWsUpstream", () => { + test("uses HTTP SSE on runtimes without a bounded response sink", async () => { + expect(bunSupportsBoundedCodexWsRelay("1.3.14")).toBe(false); + expect(bunSupportsBoundedCodexWsRelay("1.4.0-canary.1")).toBe(false); + expect(bunSupportsBoundedCodexWsRelay("garbage")).toBe(false); + expect(bunSupportsBoundedCodexWsRelay("1.4.0")).toBe(true); + expect(bunSupportsBoundedCodexWsRelay("1.5.0")).toBe(true); + expect(bunSupportsBoundedCodexWsRelay({ + version: "1.4.0", + versionWithSha: "v1.4.0 (0123abcd)", + })).toBe(true); + expect(bunSupportsBoundedCodexWsRelay({ + version: "1.4.0", + versionWithSha: "v1.4.0-canary.1 (0123abcd)", + })).toBe(false); + expect(bunSupportsBoundedCodexWsRelay({ + version: "1.4.0", + versionWithSha: "v1.5.0 (0123abcd)", + })).toBe(false); + expect(bunSupportsBoundedCodexWsRelay({ + version: "1.4.0", + versionWithSha: "malformed", + })).toBe(false); + expect(bunSupportsBoundedCodexWsRelay()).toBe( + bunSupportsBoundedCodexWsRelay(currentBunRuntimeIdentity()), + ); + if (Bun.version_with_sha.includes("-")) { + expect(bunSupportsBoundedCodexWsRelay()).toBe(false); + } + expect(rawShouldUseCodexWsUpstream(CODEX_URL, streamingInit(), "1.3.14")).toBe(false); + + const sentinel = new Response("http-sse"); + const response = await rawCodexWsUpstreamFetch( + CODEX_URL, + streamingInit(), + (async () => sentinel) as typeof fetch, + "1.3.14", + ); + expect(response).toBe(sentinel); + expect(FakeWebSocket.instances).toHaveLength(0); + + const canarySentinel = new Response("canary-http-sse"); + const canaryResponse = await rawCodexWsUpstreamFetch( + CODEX_URL, + streamingInit(), + (async () => canarySentinel) as typeof fetch, + { version: "1.4.0", versionWithSha: "v1.4.0-canary.1 (0123abcd)" }, + ); + expect(canaryResponse).toBe(canarySentinel); + expect(FakeWebSocket.instances).toHaveLength(0); + }); + test("matches only streaming POSTs to the Codex backend", () => { expect(shouldUseCodexWsUpstream(CODEX_URL, streamingInit())).toBe(true); // Non-streaming turns keep HTTP: the WS path only speaks the event protocol. @@ -87,9 +162,11 @@ class FakeWebSocket { } const RealWebSocket = globalThis.WebSocket; +const RealFetch = globalThis.fetch; afterEach(() => { globalThis.WebSocket = RealWebSocket; + globalThis.fetch = RealFetch; FakeWebSocket.instances = []; FakeWebSocket.script = () => {}; }); @@ -100,6 +177,25 @@ function installFake(script: (ws: FakeWebSocket) => void) { } describe("providerFetch routing", () => { + test("a canary runtime identity cannot open the WS transport", async () => { + const sentinel = new Response("base"); + let baseCalls = 0; + const provider = { + fetch: (async () => { + baseCalls += 1; + return sentinel; + }) as typeof fetch, + } as OcxProviderConfig; + const wrapped = providerFetch(provider, { + version: "1.4.0", + versionWithSha: "v1.4.0-canary.1 (0123abcd)", + }); + + expect(await wrapped(CODEX_URL, streamingInit())).toBe(sentinel); + expect(baseCalls).toBe(1); + expect(FakeWebSocket.instances).toHaveLength(0); + }); + test("routes eligible Codex streaming turns to WS and everything else to the base fetch", async () => { installFake(ws => { ws.emit("open", {}); @@ -113,7 +209,7 @@ describe("providerFetch routing", () => { return sentinel.clone(); }) as unknown as typeof fetch, } as unknown as OcxProviderConfig; - const wrapped = providerFetch(provider); + const wrapped = providerFetch(provider, BOUNDED_WS_RUNTIME); // Eligible: WS adapter serves it, base fetch untouched. const wsResponse = await wrapped(CODEX_URL, streamingInit()); @@ -132,6 +228,112 @@ describe("providerFetch routing", () => { }); }); +describe("handleResponses Codex WS relay selection", () => { + function forwardConfig(): OcxConfig { + return { + port: 0, + defaultProvider: "openai", + streamMode: "legacy-tee", + providers: { + openai: { + adapter: "openai-responses", + baseUrl: "https://chatgpt.com/backend-api/codex", + authMode: "forward", + codexAccountMode: "direct", + }, + }, + } as OcxConfig; + } + + function request(): Request { + return new Request("http://localhost/v1/responses", { + method: "POST", + headers: { "content-type": "application/json", authorization: "Bearer test" }, + body: JSON.stringify({ model: "gpt-5.6-luna", input: "hello", stream: true }), + }); + } + + test("a successful WS upgrade bypasses the configured legacy tee path", async () => { + installFake(ws => { + ws.emit("open", {}); + ws.emit("message", { + data: JSON.stringify({ type: "response.completed", response: { id: "r1", status: "completed", output: [] } }), + }); + }); + + const response = await handleResponses(request(), forwardConfig(), { model: "", provider: "" }, { + codexWsRuntimeIdentity: BOUNDED_WS_RUNTIME, + }); + + expect(response.status).toBe(200); + expect(FakeWebSocket.instances).toHaveLength(1); + expect(isEagerRelaySseResponse(response)).toBe(true); + const text = await response.text(); + expect(text).toContain("response.completed"); + expect(text).toContain("data: [DONE]"); + }); + + test("an HTTP fallback remains on the configured legacy tee path", async () => { + installFake(ws => ws.close()); + globalThis.fetch = (async () => new Response( + `event: response.completed\ndata: ${JSON.stringify({ + type: "response.completed", + response: { id: "r-http", status: "completed", output: [] }, + })}\n\n`, + { status: 200, headers: { "content-type": "text/event-stream" } }, + )) as typeof fetch; + + const response = await handleResponses(request(), forwardConfig(), { model: "", provider: "" }, { + codexWsRuntimeIdentity: BOUNDED_WS_RUNTIME, + }); + + expect(FakeWebSocket.instances).toHaveLength(1); + expect(isEagerRelaySseResponse(response)).toBe(false); + expect(await response.text()).toContain("response.completed"); + }); + + test("a WS queue overflow fails closed through the bounded eager relay", async () => { + const delta = "x".repeat(Math.floor(MAX_CODEX_WS_QUEUE_BYTES / 3)); + installFake(ws => { + ws.emit("open", {}); + for (let index = 0; index < 4; index += 1) { + ws.emit("message", { + data: JSON.stringify({ type: "response.output_text.delta", delta, index }), + }); + } + }); + + const response = await handleResponses(request(), forwardConfig(), { model: "", provider: "" }, { + codexWsRuntimeIdentity: BOUNDED_WS_RUNTIME, + }); + + expect(isEagerRelaySseResponse(response)).toBe(true); + const text = await response.text(); + expect(text).toContain("event: response.failed"); + expect(text).toContain("data: [DONE]"); + expect(FakeWebSocket.instances[0].closed).toBe(true); + }); + + test.skipIf(bunSupportsBoundedCodexWsRelay())( + "an older runtime stays on HTTP SSE without opening a WebSocket", + async () => { + globalThis.fetch = (async () => new Response( + `event: response.completed\ndata: ${JSON.stringify({ + type: "response.completed", + response: { id: "r-old", status: "completed", output: [] }, + })}\n\n`, + { status: 200, headers: { "content-type": "text/event-stream" } }, + )) as typeof fetch; + + const response = await handleResponses(request(), forwardConfig(), { model: "", provider: "" }); + + expect(FakeWebSocket.instances).toHaveLength(0); + expect(isEagerRelaySseResponse(response)).toBe(false); + expect(await response.text()).toContain("response.completed"); + }, + ); +}); + describe("codexWsUpstreamFetch", () => { test("relays event frames as an SSE response and sends one response.create frame", async () => { installFake(ws => { @@ -146,6 +348,7 @@ describe("codexWsUpstreamFetch", () => { expect(response.status).toBe(200); expect(response.headers.get("content-type")).toContain("text/event-stream"); + expect(isCodexWsUpstreamResponse(response)).toBe(true); const text = await response.text(); // WS-only frames are dropped so clients see the exact SSE surface they always got. expect(text).not.toContain("codex.rate_limits"); @@ -175,6 +378,23 @@ describe("codexWsUpstreamFetch", () => { expect(frame.type).toBe("response.create"); }); + test("relays an upstream error frame and closes the stream", async () => { + installFake(ws => { + ws.emit("open", {}); + ws.emit("message", { + data: JSON.stringify({ type: "error", error: { message: "upstream refused the turn" } }), + }); + }); + const response = await codexWsUpstreamFetch(CODEX_URL, streamingInit(), (() => { + throw new Error("fallback must not run after open"); + }) as unknown as typeof fetch); + + const text = await response.text(); + expect(text).toContain("event: error"); + expect(text).toContain("upstream refused the turn"); + expect(FakeWebSocket.instances[0].closed).toBe(true); + }); + test("falls back to the HTTP fetch when the upgrade is rejected before open", async () => { installFake(ws => ws.close()); const sentinel = new Response("sse-fallback", { status: 429 }); @@ -186,6 +406,7 @@ describe("codexWsUpstreamFetch", () => { const response = await codexWsUpstreamFetch(CODEX_URL, streamingInit(), fallback); // The real HTTP status must reach the existing refresh/rotation handlers. expect(response).toBe(sentinel); + expect(isCodexWsUpstreamResponse(response)).toBe(false); expect(fallbackCalls).toBe(1); }); @@ -206,6 +427,7 @@ describe("codexWsUpstreamFetch", () => { const response = await responsePromise; expect(response).toBe(sentinel); + expect(isCodexWsUpstreamResponse(response)).toBe(false); expect(fallbackCalls).toBe(1); expect(FakeWebSocket.instances[0].closed).toBe(true); } finally { @@ -229,6 +451,7 @@ describe("codexWsUpstreamFetch", () => { // pre-stream HTTP error/refresh/failover machinery. const response = await codexWsUpstreamFetch(CODEX_URL, streamingInit(), fallback); expect(response).toBe(sentinel); + expect(isCodexWsUpstreamResponse(response)).toBe(false); expect(fallbackCalls).toBe(1); expect(FakeWebSocket.instances[0].closed).toBe(true); }); @@ -247,6 +470,54 @@ describe("codexWsUpstreamFetch", () => { await expect(response.text()).rejects.toThrow("closed before a Responses terminal event"); }); + test("rejects an oversized upstream frame before parsing or enqueueing it", async () => { + installFake(ws => { + ws.emit("open", {}); + ws.emit("message", { data: "x".repeat(MAX_CODEX_WS_FRAME_BYTES + 1) }); + }); + const response = await codexWsUpstreamFetch(CODEX_URL, streamingInit(), (() => { + throw new Error("fallback must not run after open"); + }) as unknown as typeof fetch); + + await expect(response.text()).rejects.toThrow("frame exceeds the response size limit"); + expect(FakeWebSocket.instances[0].closed).toBe(true); + }); + + test("rejects a raw frame whose SSE envelope would exceed the shared frame limit", async () => { + const type = "response." + "x".repeat(64); + const base = JSON.stringify({ type, padding: "" }); + const text = JSON.stringify({ type, padding: "x".repeat(MAX_CODEX_WS_FRAME_BYTES - base.length) }); + expect(new TextEncoder().encode(text).byteLength).toBe(MAX_CODEX_WS_FRAME_BYTES); + installFake(ws => { + ws.emit("open", {}); + ws.emit("message", { data: text }); + }); + const response = await codexWsUpstreamFetch(CODEX_URL, streamingInit(), (() => { + throw new Error("fallback must not run after open"); + }) as unknown as typeof fetch); + + await expect(response.text()).rejects.toThrow("frame exceeds the response size limit"); + expect(FakeWebSocket.instances[0].closed).toBe(true); + }); + + test("disconnects an upstream that fills the bounded response queue", async () => { + const delta = "x".repeat(Math.floor(MAX_CODEX_WS_QUEUE_BYTES / 3)); + installFake(ws => { + ws.emit("open", {}); + for (let index = 0; index < 4; index += 1) { + ws.emit("message", { + data: JSON.stringify({ type: "response.output_text.delta", delta, index }), + }); + } + }); + const response = await codexWsUpstreamFetch(CODEX_URL, streamingInit(), (() => { + throw new Error("fallback must not run after open"); + }) as unknown as typeof fetch); + + await expect(response.text()).rejects.toThrow("buffered queue limit"); + expect(FakeWebSocket.instances[0].closed).toBe(true); + }); + test("a mid-stream drop surfaces as a synthesized failed terminal through the passthrough relay", async () => { const { relaySseWithFailedTail } = await import("../src/server/relay"); installFake(ws => { @@ -311,4 +582,19 @@ describe("codexWsUpstreamFetch", () => { controller.abort(); await expect(promise).rejects.toThrow(); }); + + test("aborting after open preserves the caller's abort reason", async () => { + installFake(ws => ws.emit("open", {})); + const controller = new AbortController(); + const response = await codexWsUpstreamFetch( + CODEX_URL, + { ...streamingInit(), signal: controller.signal }, + (() => { throw new Error("fallback must not run"); }) as unknown as typeof fetch, + ); + + controller.abort(new Error("turn cancelled")); + + await expect(response.text()).rejects.toThrow("turn cancelled"); + expect(FakeWebSocket.instances[0].closed).toBe(true); + }); });