fix(auth): keep long-lived clients current and coalesce token refreshes - #1102
Merged
Conversation
A `railway ca` or `railway code` session left open long enough filled the screen with "timed out waiting for config lock; proceeding without it". Three things compounded to produce it. The TUI builds one authorized client at startup and keeps it for the whole run, and `GQLClient::new_authorized` bakes the bearer into that client's default headers. Once the access token expires, every request it sends carries a dead one — forever, because the 401 handler builds a throwaway client for its retry and leaves the long-lived one just as stale. Every one of those 401s then forced a token refresh while holding the config file lock. `fs2` locks are per open file description, so the TUI's own tasks contended exactly as separate processes would: a burst of refusals queued on the lock, each waiter sat through the earlier ones' token round-trips, blew the lock's 10s timeout, and printed. Into a raw-mode alternate screen with no cursor discipline, which is why it smeared. Worse than the spam: `probe_session_liveness` refreshed unconditionally, with no "someone already did it" check. Backboard rotates and reuse-detects the refresh token on every refresh, so N simultaneous 401s meant N rotations, N-1 of them presenting an already-consumed token — grounds for revoking the whole grant. A hard logout caused by nothing but our own concurrency. The new burst test fails with 8 rotations if the coalescing is removed. So: - Set the bearer per request from the config on disk, and refresh before sending rather than after being refused. Long-lived clients now pick up refreshes instead of going permanently stale. `reqwest` applies a default header only when the request has not already set that key. - Add an in-process refresh gate ahead of the file lock, plus a 10s window in which a forced refresh stands down because one just happened. At most one task waits on the file lock, and a burst costs one rotation. The gate wait is capped at 15s: a hung token endpoint can hold it for the full 90s retry budget, and stalling every other request behind that would be worse than what it replaced. Waiters proceed with stored credentials and deliberately do not refresh on their own. - Hold warnings back while a TUI owns the terminal and flush them when it exits, deduplicated with a repeat count. Wired into `set_terminal_owned`, so every TUI gets it from its existing `restore_terminal`. Five other TUIs never set that flag and could be smeared the same way; they do now. `post_graphql_public` is new: `post_graphql` now attaches a bearer, and a `reqwest::Client` cannot be asked whether it was built with one, so the template lookup/search queries that must go out unauthenticated say so at the call site. The existing test asserting public template lookups send no auth headers is what caught this. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
A long-running
railway ca/railway codesession fills the screen with:Three causes:
GQLClient::new_authorizedbakes the token intodefault_headers; the TUI holds one client for its whole run. Once the token expires, every request 401s — permanently, since the 401 retry builds a throwaway client and leaves the long-lived one stale.fs2locks are per open file description, so the TUI's own tasks contend like separate processes. Bursts (25s auto-refresh, 1.5s watch ticks, per-env sweeps) queued on the lock until the 10s timeout fired, printing into a raw-mode alternate screen.probe_session_livenessrefreshed unconditionally, bypassing theAlreadyFreshcheck. Backboard rotates and reuse-detects the refresh token on every refresh, so N simultaneous 401s = N rotations, N-1 presenting a consumed token → grounds for revoking the whole grant.Cause 3 is a hard logout caused by our own concurrency, not just cosmetic. Disabling the new coalescing makes the burst test report the real number:
Fix
authorizationper request from disk; runensure_valid_tokenbefore sending, not after being refused. reqwest applies a default header only when the request hasn't set that key, so long-lived clients pick up refreshes. Fast path is a timestamp compare on an already-read config — no new I/O.reporter::warndefers output while a TUI owns the terminal, flushing on exit with a repeat count. Wired intoset_terminal_owned, so all TUIs get it from their existingrestore_terminal. Five other TUIs never set that flag; they do now.New:
post_graphql_public. Sincepost_graphqlnow attaches a bearer and areqwest::Clientcan't be asked whether it was built with one, the template lookup/search queries that must go out unauthenticated declare it at the call site. The existing "public template lookup sends no auth headers" test caught this.Testing
1122 tests pass, 5 consecutive clean runs, 0 clippy errors.
whoamiandtemplates searchverified against the real API.New in
auth_sim.rs:MockBackboardnow records each request'sauthorizationheader.Risks
ca. Main risk; worth a canary.OAuthInsufficientGrantinstead of "log in again". Self-heals. Easy to tighten.post_graphql_publicis opt-in, no compile-time guard. A future public call site that forgets it sends a redundant bearer. Contained — all ~230 GraphQL call sites targetget_backboard(), so it's never a cross-host leak. A newtype overreqwest::Clientwould make it checkable; out of scope here.🤖 Generated with Claude Code