Skip to content

fix(openai): merge additional_params tools into chat completions tool list; record error-envelope coverage - #2294

Merged
gold-silver-copper merged 1 commit into
mainfrom
cassette-bug-hunt-20260812
Aug 12, 2026
Merged

fix(openai): merge additional_params tools into chat completions tool list; record error-envelope coverage#2294
gold-silver-copper merged 1 commit into
mainfrom
cassette-bug-hunt-20260812

Conversation

@gold-silver-copper

Copy link
Copy Markdown
Contributor

Summary

Live-traffic bug hunt against the OpenAI, Anthropic, and Gemini providers: new cassette-backed tests were recorded against the real APIs until a genuine defect surfaced; the defect is fixed here with a cassette-backed regression test, and the passing coverage recorded along the way ships too.

Bug found and fixed

OpenAI Chat Completions: additional_params["tools"] silently replaced builder-registered tools (issue #1890's chat-path half).

  • Symptom (reproduced live, recorded): building a request with .tool(...) plus additional_params(json!({"tools": [...]})) sent only the additional_params tools to the wire. With tool_choice: required and a prompt demanding the builder tool, the model was forced to call the wrong tool — the recorded pre-fix traffic shows the request body containing only the raw tools array, and the model answering with lookup_beta when told to call lookup_alpha.
  • Root cause: providers/openai/completion/mod.rs flattens additional_params into the serialized CompletionRequest. The body is built via serde_json::to_value, where the flattened tools key wins over the typed tools field — a silent last-write-wins clobber. Every sibling path already merges instead: the Responses API (responses_api/mod.rs), Anthropic (build_tool_definitions), and Gemini (extract_tools_from_additional_params). The chat path was the one hole.
  • Fix: extract tools from additional_params in TryFrom<OpenAIRequestParams> for CompletionRequest and append it to the typed tool list, mirroring the Responses-path semantics including the loud RequestError on a malformed payload.
  • Proof: tests/providers/openai/cassette/additional_params_tools.rs + tests/cassettes/openai/additional_params_tools/builder_tools_survive_additional_params_tools.yaml — a fresh post-fix recording whose request body carries both tools and whose response calls lookup_alpha. The cassette harness matches the outbound body, so any regression back to the clobber fails replay as a mock miss.

New coverage recorded (independent of the fix)

Error-envelope preservation had zero non-2xx interactions in any committed cassette. Six new recordings (unary + streaming per provider) now pin that a real provider error keeps its status and raw body recoverable through provider_response_status() / provider_response_body():

  • openai/error_envelope/* — Responses API, 400 model_not_found (note: OpenAI returns 400, not 404, for an unknown model)
  • anthropic/error_envelope/* — Messages API, 404 not_found_error
  • gemini/error_envelope/* — generateContent + streamGenerateContent, 404 NOT_FOUND (the streaming error arrives on an SSE-content-type response and still surfaces correctly)

These also document that the streaming paths surface the pre-stream HTTP error lazily (from the first poll of the stream), which the tests capture.

Tracker sweep

All 72 open issues and 52 open PRs were read in full (including comments). Classification:

Addressed here

  • feat: support provider-hosted tools in AgentBuilder end-to-end #1890 (chat-path half): fixed as above. The Responses-path half (hosted tools via additional_params) was found to already merge correctly on main (responses_api/mod.rs removes and merges the key); the issue's remaining ask — first-class provider_tools() consumption — is a feature, not fixed here.

In scope, reproduced or verified, left as follow-ups (open fix PRs exist or fix is a larger surface)

Adjacent (PRs touching these providers that would benefit from recorded regression tests, not added here to avoid churn against their in-flight diffs): #2243, #2234, #1905, #1867, #1185, #2121, #2190, #1279.

Out of scope: all remaining open items are other providers, features, docs, or infra — full classification list available in the sweep notes; representative: #2291/#2290 (Mistral), #2241/#2263 (Cohere), #2156 (Ollama), #2292 (CI), #2284/#2278 (agent surface), #2229/#2228 (refactors), etc.

Recording matrix

Scenario Provider Kind Outcome
error_envelope unary openai, anthropic, gemini coverage gap recorded, pass
error_envelope streaming openai, anthropic, gemini coverage gap recorded, pass
additional_params tools clobber openai (chat) tracker #1890 bug reproduced live → fixed → re-recorded, pass

Cassettes: 7 newly recorded, 0 re-recorded (the fix changes the wire body only for requests carrying additional_params.tools, and no pre-existing cassette does), 0 touched otherwise. Hygiene review done file-by-file: keys/request_ids redacted by the scrubber, no secrets, no unrelated captures, placeholders redact rather than fabricate.

Verification

  • cargo fmt — clean
  • cargo clippy --all-targets --all-features — clean (no errors, no new warnings)
  • cargo test -p rig-core --all-features — 1325 lib tests + doctests pass (includes a new unit test for the mixed function/native additional_params.tools split, which also keeps Groq's compound_custom native-tool folding intact)
  • Replay suites (--test-threads=1): openai 108 passed / anthropic 79 passed / gemini 131 passed (baselines before any change: 105 / 77 / 129 passed, all green — no pre-existing failures)

Estimated API spend

Under $0.05 total: six error-envelope recordings are rejected requests (no tokens billed), and the two gpt-4o-mini tool-call recordings used ~100 tokens each.

… list (#1890)

The Chat Completions request flattens additional_params into the
serialized body via serde_json::to_value, so a raw `tools` array
supplied there silently replaced the builder's typed function tools
(flattened key wins last). Every sibling path — the Responses API,
Anthropic, and Gemini — already merges instead. Function-shaped entries
now merge into the typed tool list; non-function entries stay behind
for provider prepare_request hooks (Groq's compound_custom folding).

Reproduced live and pinned with a recorded cassette: pre-fix, a request
with tool_choice=required and a builder tool carried only the
additional_params tool on the wire, forcing the model to call the wrong
tool.

Also adds the first non-2xx recordings in the suite: six error-envelope
cassettes (unary + streaming for OpenAI, Anthropic, Gemini) proving
provider_response_status()/provider_response_body() preserve real
provider error envelopes, including the lazily-surfaced streaming case.
@gold-silver-copper
gold-silver-copper added this pull request to the merge queue Aug 12, 2026
Merged via the queue into main with commit 923e7fb Aug 12, 2026
13 checks passed
This was referenced Aug 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant