fix(mistral): three more provider bugs found by live cassette recording - #2357
Open
gold-silver-copper wants to merge 9 commits into
Open
fix(mistral): three more provider bugs found by live cassette recording#2357gold-silver-copper wants to merge 9 commits into
gold-silver-copper wants to merge 9 commits into
Conversation
Mistral's reasoning (magistral-class) models do not answer with a string.
They answer with the chunk array their reasoning docs describe: a `thinking`
chunk carrying the trace, then the answer's `text` chunk. Rig read that array
with a text-only join, on both transports, so every part not tagged `text` was
discarded and no Mistral turn could ever produce an
`AssistantContent::Reasoning` — a slot rig has and populates for anthropic,
cohere, deepseek and openrouter.
Three consequences. The trace was lost. A turn whose `max_tokens` cap was
spent inside the trace was lost *entirely*, because the thinking chunk is then
all the response carries. And the turn could not be replayed: Mistral's docs
require the whole assistant message, ThinkChunk included, be sent back into the
next request, while `finalize_request_body` deleted the shared conversion's
`reasoning_content` outright — so rig could not round-trip a reasoning block it
had itself produced.
Reading it: `Message::Assistant`'s `content` becomes
`AssistantMessageContent { text, reasoning }`, which decodes both halves and
writes both back; it derefs and displays to the answer text, so reads are
unchanged. `normalize` emits the trace as a leading reasoning block, the order
Mistral sends it in.
Reading it while streaming: the shared `delta.content` deserializer splits the
array into text and thinking, and the thinking half joins the existing
reasoning slot behind `reasoning_content` and `reasoning`. Additive for every
other OpenAI-compatible provider: a `thinking` part previously decoded to
neither, and no other in-tree provider sends one.
Writing it back: `splice_reasoning_into_content` moves an assistant message's
`reasoning_content` into its content as Mistral's thinking chunk, ahead of the
text and before the content is normalized, so the same chunk renderer handles
it and finalizing an already-finalized body stays a no-op.
One deliberate behaviour change: Mistral answers a reasoning input sent to a
model without the capability with `400 Reasoning input is not enabled for this
model`. Rig used to strip the trace, so that request silently succeeded; it
now fails with the provider's own message.
#2337 gave `mistral-embed` a declared width and deliberately left Codestral's at `None`, on the grounds that its width is configurable — pinned by a unit test asserting exactly that. But configurable is not unknown: a request naming no dimension returns 1536-wide vectors, verified live. The shared constructor falls back to `default_ndims` and then `unwrap_or_default`, so `client.embedding_model(CODESTRAL_EMBED).ndims()` was 0 — not a width a vector store can size itself from, which is the reason #2337 gave for fixing mistral-embed. Both Codestral ids now declare 1536. An explicit width still governs and still rides Mistral's own `output_dimension` spelling; the echoed default is a value Mistral documents and accepts.
Mistral was the only provider whose transcription hand-rolled the send/status/decode tail instead of using the shared driver, and the tail it built ended in a bare `from_http_response(status, body)` with no `with_response_headers`. rig#2210 states the contract that a failed capability call preserves its response headers so a caller can read `Retry-After`; every other transcription implementation honours it through `send_transcription` / `send_json_transcription`, both of which take the response apart with `into_parts`, guarded by a dedicated header-preservation test module. Routed through `send_transcription` like every other multipart transcription. The usage log moves onto the response's `TryFrom`, so it still fires. Scope: the bundled reqwest client raises a non-2xx as `InvalidStatusCodeWithDetails`, which already carries its headers, so this is unreachable on that transport. It reproduces on any custom `HttpClientExt` that hands the non-success response back — the transport shape the shared drivers' own rig#2210 tests use.
Migrates the `#[ignore]`d live smoke test — the last Mistral capability with no fixture directory at all — into recorded cells, and extends it into the matrix for the header-preservation fix: both Voxtral models, a populated `segments` array behind `timestamp_granularities`, a language hint, an unknown model's 4xx and a bogus key's 401. The regression cell itself is a unit test rather than a cassette, and the matrix says why: only a transport that hands a non-success *response* back can lose the headers, and the cassette proxy is driven through reqwest, which takes the other branch. The outbound multipart form is likewise unit-pinned — a multipart request is exported with no body at all, so a fixture cannot express it.
The declared width against the vectors Mistral actually returns, for both
Codestral ids and both request classes (no dimension, an explicit one), plus a
batch cell proving every row of a batch is the declared width and a
mistral-embed control proving the fixed-width model is untouched.
The input space is small and fully enumerable — {model} × {width the caller
asks for} — so the matrix is written out in full in the file's header, with the
cells that need no traffic (a width rejected before the wire) named as the unit
tests they are.
Twenty-six recorded cells over the reasoning surface: both transports, raw model and agent, trace-plus-answer and trace-alone and trace-beside-a-tool-call, structured output, the magistral-branded model id, the replay of the trace into the following request read off the recorded request bytes, and the three error shapes (a reasoning history on a model without the capability, an unsupported effort value, a bogus key). Three of them are controls that record a turn with *no* thinking chunk, so the rest are provably about the chunk rather than about array content in general. Every cell checks its own premise against its own recorded bytes — a cell whose provider turn stopped carrying a thinking chunk fails rather than passing while covering nothing. The premise reads are skipped while recording, because the fixture is written after the test body returns. Mistral also gains an entry in the cross-provider reasoning contract (`tests/common/reasoning.rs`) that every other reasoning-capable provider in the tree already runs and Mistral had none of: `reasoning_roundtrip` and `reasoning_tool_roundtrip`, streaming and non-streaming, including the agent loop whose chat-history assertion requires a reasoning block.
… fixes The reasoning fix changes a public provider wire type and one silent behaviour, so it gets a `MIGRATING.md` section for each: the assistant `content` field's new type (with the deref that keeps every read compiling), and the 400 Mistral now answers a reasoning history sent to a model without the capability.
`cargo doc --all-features` runs with `RUSTDOCFLAGS=-D warnings` in CI, and `rustdoc::private_intra_doc_links` fires on the `Serialize` impl's reference to `splice_reasoning_into_content`. The name is still there, as code rather than a link, with its visibility stated.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Round 4 of the Mistral cassette hunt. #2331 and #2337 landed twelve bugs across three sweeps, the third of which came up dry. This one went after the ground those sweeps left unrecorded — the
#[ignore]d live tests with no fixture directory — and found three more, one of them a whole capability's worth of data loss.codestral-embedreportsndims() == 0while returning 1536-wide vectorsRetry-Afteris unreachable42 recorded cassettes + 27 unit cells. 26 of the 42 fail on
origin/mainandpass here — the other 16 are controls (a turn with no thinking chunk, a
traceless history, the fixed-width embedding model) and cells that characterise
Mistral's own behaviour rather than rig's, and they pass on both sides by
design.
Bug 13 — a reasoning turn's trace is dropped, and cannot be replayed
Mistral's reasoning models do not answer with a string. They answer with the
chunk array their reasoning docs describe: a
thinkingchunk carrying thetrace, then the answer's
textchunk. Live,mistral-small-latestwithreasoning_effort: "high":Rig read that array with a text-only join, on both transports:
Every part that is not tagged
textis discarded, so the trace never reachedAssistantContent::Reasoning— a slot rig has and populates for anthropic,cohere, deepseek and openrouter. Mistral had no code path that could
construct one.
Three consequences, each worse than the last:
max_tokenscap is spentinside the trace, the thinking chunk is all the response carries — so
after the text-only join the turn had no content at all, and the
Lengthfinish reason was the only thing left.
assistant message, ThinkChunk included, be sent back into the next request.
Rig did the opposite: the shared OpenAI-compatible conversion renders a
Reasoningblock as areasoning_contentstring, andMistralExt::finalize_request_bodydeleted the field(
message.remove("reasoning_content"), client.rs:139) — so rig could notround-trip a block it had itself produced.
Consequence 3 is the one whose severity I could not pin down, and I want to
be precise about it. Two hand-run live pairs had the traceless replay re-issue
the call it already made instead of answering from the tool result — the
incoherence Mistral's docs warn about:
But a third run through the same request shape answered correctly, so it is not
reproducible, and nothing in this PR asserts it. The cell that used to has
been rewritten as a request-bytes control, and its doc says why. What stands on
its own without that anecdote: the trace is data the API returned and rig
dropped, a truncated turn came back with nothing at all, Mistral's docs require
the replay, and rig could not round-trip a reasoning block it had itself
produced.
The streaming half is the same defect on the same helper: thinking arrives as
delta.contentarrays, whichdeserialize_delta_contentjoined for text andtherefore dropped.
Worth stating because it was the worse possibility: the trace is not
spliced into the answer. It is dropped, not corrupted — the answer's deltas
arrive as plain strings on their own.
The fix
Reading it (Mistral-local).
Message::Assistant'scontentbecomesAssistantMessageContent { text, reasoning }, which decodes both halves of thearray and writes both back. It
Derefs andDisplays to the answer text, soevery read of the field is unchanged; only a struct literal needs
.into().normalizeemits the trace as a leadingAssistantContent::Reasoning, theorder Mistral sends it in and the order the streamed twin emits.
Reading it (shared). The streamed
delta.contentarray is now split intotext and thinking by
DeltaContentin the OpenAI-compatible streaming layer,and the thinking half joins the existing reasoning slot behind
reasoning_contentandreasoning. Blast radius: every OpenAI-compatibleprovider rides that deserializer. The change is additive — a
thinkingpartpreviously decoded to neither text nor reasoning — and no other in-tree
provider sends one: the only cassettes carrying
"type":"thinking"areanthropic's (30) and cohere's (2), and neither rides this wire. No fixture
outside Mistral needed re-recording.
Writing it back (Mistral-local).
splice_reasoning_into_contentmoves anassistant message's
reasoning_contentinto itscontentas Mistral'sthinking chunk, ahead of the text, before the content is normalized — so the
trace is rendered by the same chunk renderer as every other part, and
into_mistral_chunkgains athinkingarm so finalizing an already-finalizedbody stays a no-op.
One deliberate behaviour change, stated plainly. Mistral answers a
reasoning input sent to a model without the capability with
400 Reasoning input is not enabled for this model(recorded). Rig used tostrip the trace, so that request silently succeeded. It now fails loudly, with
the provider's own message. That is the trade: a loud, actionable rejection on
the models that cannot take a trace, in exchange for coherence on the ones that
can.
MIGRATING.mdsays so and names the workaround.Matrix —
tests/cassettes/mistral/reasoning_content/+ the shared suites26 cells in
reasoning_content/, plus 4 in the cross-provider reasoningcontract (
reasoning_roundtrip/,reasoning_tool_roundtrip/) that Mistral hadno entry in before, because it produced no reasoning to test. Dimensions
crossed: transport (blocking × streaming) × surface (raw model × agent) ×
content shape (trace + answer, trace alone, trace beside a tool call, trace
beside a
json_schemaformat) × model id (mistral-small-latest,magistral-small-latest, a model without the capability) × direction (whatMistral sends, what rig sends back).
origin/mainblocking_keeps_the_reasoning_traceCompletionModelblocking_without_reasoning_effort_has_no_traceCompletionModelblocking_reasoning_effort_none_is_a_plain_stringCompletionModeleffort: noneblocking_truncated_thinking_still_yields_the_traceCompletionModelblocking_reasoning_beside_a_tool_callCompletionModelblocking_reasoning_on_the_magistral_aliasCompletionModelblocking_agent_chat_keeps_the_trace_in_historyAgent::chatblocking_reasoning_with_structured_outputCompletionModeljson_schemastreaming_yields_the_reasoning_traceCompletionModelstreaming_without_reasoning_effort_yields_no_traceCompletionModelstreaming_reasoning_beside_a_tool_callCompletionModelstreaming_truncated_thinking_yields_the_traceCompletionModelstreaming_reasoning_on_the_magistral_aliasCompletionModelstreaming_agent_stream_keeps_the_traceAgent::stream_promptstreaming_reasoning_with_structured_outputCompletionModeljson_schemastreaming_terminal_aggregates_the_traceCompletionModelroundtrip_replays_the_thinking_chunkCompletionModelroundtrip_replays_the_trace_beside_a_tool_callCompletionModelroundtrip_without_a_trace_sends_a_plain_stringCompletionModelroundtrip_lets_the_model_answer_from_the_tool_resultCompletionModelroundtrip_a_traceless_tool_history_sends_no_thinking_chunkCompletionModelroundtrip_replays_a_trace_captured_from_a_streamCompletionModelreasoning_history_on_a_model_without_the_capability_is_rejectedCompletionModelan_unsupported_reasoning_effort_keeps_the_id_and_bodyCompletionModelbogus_key_reasoning_request_keeps_the_id_and_bodyCompletionModelparity_blocking_and_streaming_both_carry_a_traceCompletionModelreasoning_roundtrip::{streaming,nonstreaming}reasoning_tool_roundtrip::{streaming,nonstreaming}Every cell checks its own premise against its own recorded bytes: a cell about
the thinking chunk fails if the recorded turn stopped carrying one, and the
controls fail if one appears. Those reads are skipped while recording,
because the fixture is written after the test body returns — and a guard
rejects a body too short to be one, which is how the SSE block-scalar parsing
bug the review found was caught.
19 unit cells next to the provider cover the schema-legal shapes no live
turn has been observed to send — a bare-string
thinkingpayload, severaltraces in one turn, a payload-less chunk, the
{"thinking": []}closing frame,the request-side splice against string/array/absent/null content, idempotence —
plus 4 in the shared streaming layer for the delta split and the
three-spelling reasoning preference.
Dropped with reason. A cell asserting what a traceless replay makes the
model do: it did not reproduce (see above), so there is no such cell.
Bug 14 —
codestral-embeddeclares zero dimensions#2337 gave
mistral-embeda declared width and deliberately left Codestral'sat
None, on the grounds that its width is configurable — pinned by a unittest asserting
default_ndims(CODESTRAL_EMBED) == None. But configurable isnot unknown: a request naming no dimension returns 1536-wide vectors.
GenericEmbeddingModel::makedoesndims.or_else(|| Ext::default_ndims(&model)).unwrap_or_default(),so
client.embedding_model(CODESTRAL_EMBED).ndims()was 0 — not a width avector store can size itself from, which is the reason #2337 gave for fixing
mistral-embed.Fix: both Codestral ids declare 1536. An explicit width still governs and
still rides Mistral's own
output_dimensionspelling — and a model asked forits own default sends no width at all, so a request that named no dimension
puts exactly the bytes on the wire it always did. That guard already existed for
the fixed-width models; it now covers Codestral too, and the recorded cell
asserts it against the request body.
Matrix —
tests/cassettes/mistral/embedding_dimensions/The input space is small and fully enumerable — {model} × {width the caller
asks for} — so it is written out in full rather than sampled. Mistral serves two
embedding models under two ids each, and
embedding_dimensionssorts arequested width into exactly three classes: none, the model's own default, and
some other value.
codestral_embed_declares_its_real_widthcodestral-embedcodestral_embed_honors_an_explicit_output_dimensioncodestral-embedcodestral_embed_dated_alias_declares_the_same_widthcodestral-embed-2505codestral_embed_batches_at_the_declared_widthcodestral-embedmistral_embed_still_declares_its_own_widthmistral-embedcodestral_embed_declares_its_default_widthcodestral_embed_sends_its_width_as_output_dimensioncodestral-embedmistral_embed_declares_its_width_without_requesting_itmistral-embeddimensionsrejectionCells 1 and 5 also assert, against the recorded request, that a call naming no
width carries neither width field.
Dropped with reason. A batch over Mistral's 256-input cap: already recorded
in
capability_edges.rs, and about chunking rather than width.Bug 15 — a failed transcription drops its response headers
Mistral was the only provider whose transcription hand-rolled the
send/status/decode tail instead of using the shared driver, and the tail it
built ended with:
No
.with_response_headers(...). Every other transcription implementation goesthrough
send_transcription/send_json_transcription, both of which takethe response apart with
into_partsand attach the headers, guarded by adedicated
header_preservation_testsmodule. rig#2210 states the contract thata failed capability call preserves its response headers so a caller can read
Retry-After; Mistral's transcription did not.Fix: route through
send_transcription, like every other multiparttranscription. The usage log moves onto the response's
TryFrom, so it stillfires.
Scope, stated precisely: the bundled reqwest client raises a non-2xx as
InvalidStatusCodeWithDetails, which already carries its headers, so this isunreachable on that transport. It reproduces on any custom
HttpClientExtthathands the non-success response back — the transport shape the shared drivers'
own rig#2210 tests use.
Matrix —
tests/cassettes/mistral/transcription/Migrated from the
#[ignore]d live smoke test this file used to hold — the lastMistral capability with no fixture directory at all.
The bug's own input space is {transport shape} × {outcome}, and only one column
can lose headers: a transport that hands a non-success response back. That is
exactly the column a recording cannot reach, because the cassette proxy is
driven through reqwest, which raises a non-2xx as an error that already carries
its headers. So the regression cell is a unit test with a recording transport,
and the recorded cells cover the capability's live surface around it.
voxtral_mini_transcribes_the_audio_fixturevoxtral-minivoxtral_small_is_not_a_transcription_modelvoxtral-smallInvalid modeltimestamp_granularities_populate_segmentsvoxtral-minisegmentsnon-emptyan_array_of_timestamp_granularities_is_rejectedvoxtral-minia_language_hint_is_acceptedvoxtral-minilanguageform fieldan_unknown_model_is_rejected_with_its_bodybogus_key_transcription_keeps_status_and_bodyvoxtral-minitranscription_non_success_preserves_response_headerstranscription_non_success_preserves_status_and_bodytranscription_form_carries_the_documented_fieldsTwo findings fell out of recording this matrix, both pinned rather than fixed:
VOXTRAL_SMALLis not a transcription model. It is declared besideVOXTRAL_MINIintranscription.rs, so it reads as the larger transcriptionmodel; the live catalog reports
audio_transcription: falsefor it (andaudio: true,completion_chat: true— it is the audio-chat model themultimodal matrix already drives), and the endpoint answers
400 Invalid model: voxtral-small-latest. The constant's doc now says so, andcell 2 backs it with the wire.
timestamp_granularitiesmust be a bare string, not an array. The sharedmultipart builder JSON-encodes a non-string
additional_paramsvalue, so["segment"]goes out as a single field whose value is["segment"]andMistral answers 422 naming the enum it wanted. Not fixed: rig has one
multipart encoding for every provider on the shared driver, the string form
works, and the error names the field. Cell 4 pins it.
Census — what the hunt looked at and did not report
n > 1selectschoices[0]positionally while streaming selectsby
index == 0(fix(mistral): eight more provider bugs found by live cassette recording #2337). Recorded a blockingn: 2turn: Mistral returnedthe candidates in index order, so the two transports agree. Not a bug on
the evidence available; a hardening note at most.
EMITS_COMPLETE_SINGLE_CHUNK_TOOL_CALLSis honoured: the flag reachesshould_emit_completed_tool_call_immediatelythrough the profile, and a liveministral-3b stream does emit a whole tool call in one chunk
(
{"name":"add","arguments":"{\"a\": 2, \"b\": 3}"}).capabilities,deprecationanddefault_model_temperature, none of whichModelhas a slot for, and nooutput-token field at all — so nothing is dropped.
max_output_tokensstaysNonebecause Mistral reports none.prompt_tokens_details.audio_tokensoutsideprompt_tokens, the same accounting fix(mistral): eight more provider bugs found by live cassette recording #2337 folded into the completion path.TranscriptionUsagetypesprompt_audio_secondsasOption<i32>while thecompletion
Usagetypes itOption<u64>; every observed value is a wholenumber of seconds and I could not force a fractional one, so this is listed
as an unconfirmed candidate rather than a bug.
reasoning_effortvocabulary:mistral-small-latestacceptsnoneandhighonly —lowis a 400 naming the two it takes. Recorded as an errorcell rather than reported as a rig defect.
Independent review
A cold review of the branch caught seven defects, all fixed here. Two were the
kind this process exists to find:
reasoning-contract streaming cell asserted
stats.reasoning_block_count > 0,but the chat-completions wire never signs or closes a reasoning part, so
MintedReasoningLifecycleemitsReasoningDeltas and a bare end thatStreamingCompletionResponsesuppresses — no completeReasoningblock isever yielded. Every other chat-completions provider's equivalent cell guards
the same claim; only the Responses-API suites assert a block. Fixed to
reasoning_delta_count > 0 || reasoning_block_count > 0, which is what thebranch's own
reasoning_content.rscell already used.The fixture reader took
body:line-prefixed scalars, but an SSE body isa YAML block scalar (
|+) whose payload is the indented lines thatfollow. So every streaming premise check would have panicked on the header
"|+", and the streaming control would have passed vacuously — the exact"passes while covering nothing" failure the premise checks exist to prevent.
The reader now folds block scalars, and a
assert_recorded_bodies_are_readableguard fails rather than letting a too-short body count as evidence.
Five more, each fixed:
carrying
"output_dimension": 1536. The fixed-width models already had aguard for exactly that echo; it now covers Codestral too, so the default
request's bytes are unchanged, and cell 1 asserts it against the recorded
request.
preference holds — reordering the preference would have left it green. The
preference is now its own function and the test drives it.
not assert that it does. It does now.
into_mistral_chunkwould render{"thinking": []}— the streaming wire'sclosing frame — as a chunk with an empty text part, contradicting the rule
the request-side splice enforces. Both paths now refuse an empty trace.
MIGRATING.mdsaid reads are unchanged without naming the.clone()case,and the CHANGELOG's 400 claim was too broad: an assistant turn carrying
only reasoning is dropped by the shared message conversion before it can be
rejected, so a truncated-thinking turn still replays as nothing. Both are
stated correctly now.
Verification
cargo fmt --all --check— cleanRUSTFLAGS="-D warnings" cargo clippy -p rig-core -p rig -p rig-agent --all-targets --all-features— cleancargo check -p rig-core --target wasm32-unknown-unknown— cleancargo doc --workspace --no-deps— one warning (unresolved link to AudioGenerationModel, inproviders/venice/mod.rs), pre-existing and untouched by this branchcargo test -p rig-core --all-features --lib— passes--test-threads=1, noMISTRAL_API_KEY): 129 passed, 0 failed, 19 ignored, against 87/20 onorigin/mainorigin/mainrepro (provider sources reset toorigin/main, same fixtures, same test code): 26 failed, 103 passed. Every failure is a cell this PR is about; the binary was verified by marker to not contain the fix before the run, so the repro cannot be a stale-build artefact.Representative on-
origin/mainoutput:and, for the replay direction, the mock miss that is the diff:
Fixtures were audited for leaked account/org identifiers: the only ids present
are the harness's
chatcmpl-REDACTED_n/req_REDACTED_n/call_REDACTED_nplaceholders.
Roughly 90 live requests were made across the hunt, all on
mistral-small-latest/ministral-3b-latest/codestral-embed/voxtral-mini-latestwith small caps;magistral-small-latestis the samemodel as
mistral-small-latestin the live catalog, so the reasoning cells costno more than the ordinary ones.