fix(clients): honour token_provider on the SSE-backed clients and never hang on a dead stream - #200
Merged
Merged
Conversation
…er hang on a dead stream
The facade accepted a `token_provider`, but only the GraphQL facades used
it: OperatorClient, OperationClient and QueryClient captured `token` /
`headers` at construction for every REST call and every SSE connect. The
backend revokes the previous JWT on each session refresh, so a rotating
credential left those three clients dead after the first rotation — and a
stream that then failed to open emitted the transport Exception, which the
dict-only `on_error` handlers choked on inside `emit`, leaving the wait
loop spinning forever. Python twin of typescript-client #206.
- token_utils.resolve_auth_headers builds request/stream headers per call:
static headers unchanged without a provider, the provider's current
credential (routed by shape) replacing any stale auth header with one.
apply_auth_header is the single routing rule; auth_integration aliases it.
- Operator/Query/Operation clients build a fresh Client and SSEConfig from
it per call and per connect.
- OperatorClient splits run errors from transport errors and, when the
stream gives no verdict, follows the run over /v1/operations/{id}/status
(poll_interval; definitive 4xx ends it, transient failures retried).
error_details passes through on every result path.
- Operator/Query handlers accept Exception payloads, register
error/max_retries_exceeded, take the cancellation payload, and raise when
the stream ends without a result instead of spinning or returning None.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Python counterpart of RoboFinSystems/robosystems-typescript-client#206. The facade already accepts a
token_provider, but only the GraphQL facades (ledger / investor / library) consulted it —OperatorClient,OperationClientandQueryClientcapturedtoken/headersonce at construction and reused them for every REST call and every SSE connect. The backend revokes the previous JWT on each session refresh, so for a rotating credential those three clients went dead after the first rotation (and a provider-only config never worked for them at all:execute_queryraisedNo API key provided).It was worse than a 401: a stream that fails to open emits the transport
Exceptionitself, and the operator / queryon_errorhandlers diderr.get(...)on it — theAttributeErrorwas swallowed byemit(which only logs),completednever flipped, and_wait_for_*spun forever.OperationClienthad already fixed this for itself; the fix never reached the other two. Theon_cancelled()handlers took no argument, so a cancellation event hung the same way.The three clients now resolve the credential per call / per connect through the same
token_providerthe GraphQL facades use, transport errors end the wait, andOperatorClientfollows a queued run over/statuswhenever the stream gives no verdict — so a run that is already executing (and billing) is never lost.Changes
Hand-written facades under
robosystems_client/clients/only; generatedapi/andmodels/untouched.token_utils.py—resolve_auth_headers(config): the staticheadersunchanged when no provider is set (today's behaviour byte-for-byte, plus the statictokenrouted by shape when the headers carry no credential); with a provider, anyX-API-Key/Authorizationin the static headers is replaced by the provider's current credential.apply_auth_headermoves here as the single routing rule;auth_integration._apply_auth_headeris now a thin alias of it.sse_client.py—event_error_message(err): text of anerrorpayload whether it is a terminal-event dict or a transportException.operator_client.py—_rest_client()/_sse_config()build a freshClient/SSEConfigper call and per connect fromresolve_auth_headers(the REST path drops theAuthenticatedClient(auth_header_name="X-API-Key")form, which sent a JWT as an API key alongside the real Bearer header)._wait_for_operator_completionsplits run errors (operation_error/operation_cancelled→ raise) from transport errors (error/max_retries_exceeded→ no verdict); with no verdict it calls_poll_for_completion, which pollsGET /v1/operations/{id}/statusuntilcompleted/failed/cancelled, ends on a definitive 4xx, retries up to three consecutive transient failures, and relays status messages toon_progress. NewOperatorOptions.poll_interval(seconds, default 2.0) andOperatorResult.error_details, passed through on the sync and stream paths as well (_operator_resultis the one mapper).close()now reaches an in-flight stream (the field it checked was never assigned).query_client.py— same_rest_client()/_sse_config()per-call resolution;on_erroracceptsExceptionpayloads;error/max_retries_exceededregistered on both waits; the wait and the streaming generator raise when the stream ends without a verdict instead of spinning or returningNone;on_cancelledtakes the event payload.operation_client.py— stream, status and cancel headers come fromresolve_auth_headersper call (sync and async monitors).self.headers/self.tokenare kept as attributes.README.md— "Rotating Credentials (token_provider)" section.test_auth_header_resolution.py(resolver, routing alias,event_error_message, operation-client headers),test_operator_client_ops.py(stream completion, provider-at-connect for stream and REST, polling fallback for a stream that cannot open and one that ends early, run error, cancellation, failed status, definitive 404, transient retry, give-up,error_details),test_query_client_sse.py(completion, transport error, retries exhausted, no verdict, cancellation, provider-at-connect; same for the streaming generator). Sync paths covered; the async operation monitor got the header change but its wait logic is unchanged and untested here.Compatibility
ADDITIVE
token_utils.resolve_auth_headers,token_utils.apply_auth_header,sse_client.event_error_message,OperatorOptions.poll_interval,OperatorResult.error_details./statusinstead of spinning; a transport error or cancellation on an operator/query stream now raises instead of hanging (or returningNone); atoken_provideris honoured by the operator / operations / query clients. With no provider, request and stream headers are exactly what they were.Testing
just test-allequivalent run in-session:ruff format --checkandruff checkclean,basedpyright0 errors,pytest558 passed / 17 skipped (35 added).