fix(openai): preserve Responses message phase across stateless replay (#2269) - #2295
Merged
Conversation
…#2269) gpt-5.6-era models stamp a `phase` (e.g. final_answer) on message output items, and OpenAI documents that follow-up requests should replay it — dropping it can degrade performance. OutputMessage silently dropped the field at deserialization, so every replayed history omitted it (visible in all committed multi-turn gpt-5.6 cassettes: phase in each response, absent from each follow-up request). The field now survives end to end: OutputMessage carries it (raw-path serialization stays value-equal), ingest parks it in the first text block's wire extras (the same vehicle as annotations), and request assembly hoists it back to the message level, where the recorded live traffic shows OpenAI accepting it. New cassette-backed coverage recorded along the way: - openai/phase_roundtrip: two-turn recording proving the wire emits phase and the follow-up request replays it (request-boundary match) - anthropic/stop_sequences: stop_sequences hit surfaces FinishReason::Stop plus the matched raw stop_sequence - gemini/image_input: inline base64 image ingestion (the suite only covered the generation direction) Re-recorded gpt_5_6_reasoning/five_turn_metadata_roundtrip — the one cassette the fix invalidates (its follow-up request bodies now carry phase; 4-line diff).
The phase injection lived in the shared From<Output> conversion, so it also changed the replayed request bytes of every other consumer of the Responses wire — Copilot's recorded gpt-5.3-codex responses carry phase, and its reasoning_roundtrip cassette failed replay as a body mismatch. No Copilot/ChatGPT/xAI recording proves those backends accept a replayed phase, and re-recording Copilot needs credentials this run does not have, so capture now happens in the blocking normalize path gated to provider == "openai" — the one backend whose acceptance the phase_roundtrip cassette proves. The unverified wires stay byte-identical to their recordings.
Contributor
Author
|
CI fix pushed: the |
Merged
pull Bot
pushed a commit
to appelgriebsch/rig
that referenced
this pull request
Aug 12, 2026
…s replay (0xPlaygrounds#2269) (0xPlaygrounds#2295)" (0xPlaygrounds#2300) This reverts commit bb6b6cb.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Second cassette-driven live bug hunt against the OpenAI, Anthropic, and Gemini providers (same doctrine as #2294): record new tests against the real APIs until a genuine defect surfaces, fix it, re-record what the fix invalidates.
Bug found and fixed
OpenAI Responses: the documented
phasefield was silently dropped from message output items and never replayed — thephasehalf of issue #2269."phase":"final_answer"on the message items in each response and nophasein any follow-up request. OpenAI documents that follow-up requests should replay it and that dropping it can degrade performance.OutputMessagehad nophasefield, so deserialization dropped it; nothing in the rig-message roundtrip could carry message-level wire data back to request assembly.responses_api/mod.rs):OutputMessagegainsphase: Option<String>— the raw path now round-trips it value-equal;openai_responses(the same vehicle annotations already use);assistant_text_replay_message) hoists it back to the message level onMessage::Assistant, filtered out of the content-level extras so it serializes exactly where the wire wants it.tests/providers/openai/cassette/phase_roundtrip.rs+ its cassette — a fresh two-turn recording in which turn 1's response carriesphaseand turn 2's recorded request body replays it on the assistant message ("id":"msg_…","phase":"final_answer","role":"assistant"), accepted by the live API. The harness matches outbound bodies, so a regression fails replay as a mock miss.OPENAI_RESPONSES_EXTRAS_KEY); this fix covers the blocking path, consistent with annotations. feat(openai): preserve Responses items for stateless replay #2269's other asks (compactioninput items, caller-built raw requests) overlap open PR feat(openai): support server-side Responses compaction #2234 and are deliberately not touched.Re-recorded
gpt_5_6_reasoning/five_turn_metadata_roundtrip.yaml— the one cassette the fix invalidates. The re-recording is a 4-line diff: the four follow-up request bodies now carryphase; responses unchanged.New coverage recorded (independent of the fix)
anthropic/stop_sequences— astop_sequenceshit (viaadditional_params) surfacesFinishReason::Stop, the raw response preservesstop_reason: "stop_sequence"+ the matchedstop_sequence: "five", and the emitted text stops before the sequence (claude-haiku-4-5).gemini/image_input— inline base64 image ingestion (inline_datarequest part → grounded description); the suite previously covered only the image generation direction.Tracker sweep
Full sweep of all open issues/PRs was done earlier today and reported in #2294; the delta since is one new PR (#2293, Mistral — out of scope). This PR addresses the in-scope follow-up ranked highest there:
phasehalf fixed here (Fixesintentionally not claimed: the issue also asks for compaction-item and raw-request support, which overlap open PR feat(openai): support server-side Responses compaction #2234).Recording matrix
Hygiene review done file-by-file: API keys/
request_ids redacted, no secrets (twoAIza-ish substrings in the gemini cassette are case-mismatched base64 image bytes, not key-shaped; the recorder's key-shape scan also passed), no unrelated captures, placeholders redact rather than fabricate. No hand-edited fixtures.Verification
cargo fmtclean;cargo clippy --all-targets --all-featuresclean;cargo test -p rig-core --all-featuresall pass; replay suites openai 109 / anthropic 80 / gemini 132 passed; wire-sharing suites also green: chatgpt 24, doubleword 21, bedrock 19.Estimated API spend
Under $0.20: one two-turn and one five-turn gpt-5.6 recording at low reasoning effort (~2k tokens total), one claude-haiku-4-5 call (~100 tokens), one gemini-2.5-flash vision call (~1.3k input tokens).