Skip to content

Retry throttled ARM Machine requests - #290

Merged
Wenxuan W (wenxuan0923) merged 1 commit into
mainfrom
wenx/retry-arm-machine-429
Aug 26, 2026
Merged

Retry throttled ARM Machine requests#290
Wenxuan W (wenxuan0923) merged 1 commit into
mainfrom
wenx/retry-arm-machine-429

Conversation

@wenxuan0923

Copy link
Copy Markdown
Collaborator

Summary

  • configure the shared ARM Machine SDK pipeline with a bounded 30-retry budget for every Machine request
  • honor ARM-provided Retry-After, Retry-After-Ms, and x-ms-retry-after-ms delays before using jittered exponential backoff
  • apply the same retry behavior to direct ARM and dev-proxy Machine clients
  • cover GET, create/update, retry-delay headers, and exhausted retry budgets

Validation

  • make check
  • go test -race ./pkg/aksmachine

Fixes #285

Copilot AI lite review requested due to automatic review settings August 26, 2026 17:16

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR hardens the AKS ARM Machine client against transient ARM throttling (HTTP 429) during bootstrap by configuring the Azure SDK retry policy to use a larger, bounded retry budget and to respect ARM-provided retry-delay headers, addressing the failure described in issue #285.

Changes:

  • Add a dedicated ARM Machine SDK retry configuration (30 retries, 2m per-try timeout, 5s base retry delay, 1m max backoff) and apply it to the standard ARM Machine client.
  • Apply the same retry configuration to the dev-proxy ARM Machine client so proxy usage matches production retry behavior.
  • Add targeted tests covering option wiring, retry on 429 for GET and create/update, honoring retry-delay headers, and stopping after the retry budget is exhausted.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

File Description
pkg/aksmachine/client_armapi.go Introduces and applies a shared ARM Machine-specific Azure SDK retry configuration.
pkg/aksmachine/client_armapi_test.go Adds tests validating retry configuration and behavior for throttled ARM Machine requests.
pkg/aksmachine/client_armapi_proxy.go Reuses the same ARM Machine retry configuration for the dev-proxy client.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@wenxuan0923
Wenxuan W (wenxuan0923) merged commit d09ccdc into main Aug 26, 2026
13 of 15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ensure-machine fails bootstrap instead of retrying transient ARM 429

3 participants