The provider probe no longer marks a healthy model family unavailable - #340
Merged
Merged
Conversation
…ailable The probe pinged each provider with max_tokens: 1. One model family rejects every spelling of a token cap, explicitly or behind the gateway's masked 400, so the ping failed twice on its own parameter and the probe reported a reachable, unlocked provider as down, with the mark on every model of the family and a banner telling the user to unlock a key that was never locked. The ping now carries no cap. Its cost is bounded another way: the response is read only until its first frame, which is proof enough that the provider is answering, and the request is then abandoned. A body that names a request parameter as the problem is read as a reachable provider whatever else it says, since a provider that argues about the request's shape is up. Tests cover the parameter rejection, the first-frame read, and the unchanged locked and unavailable verdicts.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The provider probe pinged with
max_tokens: 1. One model family rejects every spelling of a token cap (max_output_tokensexplicitly,max_tokensandmax_completion_tokensbehind the gateway's masked 400; measured earlier ongpt-5.6-sol), so the ping failed twice on its own parameter and the probe reported a reachable, unlocked provider as down. Every model of the family showed[unavailable]and the banner told the user to unlock a key that was never locked.The ping now carries no cap. Cost is bounded by reading the streamed response only to its first frame, which is proof the provider answers, then aborting. A body that names a request parameter as the problem is classified as a reachable provider regardless of status, since a provider arguing about the request's shape is up. Locked and unavailable verdicts are unchanged.
Verified live against the cap-rejecting family on the gateway: the verdict is now ok. Tests: parameter rejection reads as up with no second ping; a healthy stream is read to one frame; existing locked, unlock-url, masked-twice, and cache cases still pass. 184 server tests pass.