Skip to content

fix(cli): harden Azure deployment safety - #531

Draft
Pal Lakatos-Toth (pallakatos) wants to merge 6 commits into
mainfrom
fix/azure-deployment-safety
Draft

fix(cli): harden Azure deployment safety#531
Pal Lakatos-Toth (pallakatos) wants to merge 6 commits into
mainfrom
fix/azure-deployment-safety

Conversation

@pallakatos

@pallakatos Pal Lakatos-Toth (pallakatos) commented Sep 2, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • remove the deprecated ACR Notary v1 trustPolicy
  • replace the hardcoded AKS 1.33 version with regional
    KubernetesOfficial discovery and validation for new clusters
  • validate generated Azure names, AKS node-resource-group length, VM SKU
    availability, total/family vCPU quota, and retained resource existence before
    billable deployment work
  • preserve existing resource-group tags and make existing AKS deployments
    reuse-only unless operators use dedicated AKS upgrade/pool workflows
  • add purge-protected Key Vault recovery and generated-resource-group-only
    rollback guarded by Azure CanNotDelete management locks
  • pin the selected subscription across up, add, upgrade, fast upgrade,
    Agent ID setup, image acquisition, sandbox bring-up, and destroy
  • keep Kubernetes-only sandbox destroy available when Azure authentication has
    expired
  • regenerate and verify npm-bundled Bicep/ARM assets

Backward compatibility

  • normal resource-group naming and permissions are unchanged unless
    --rollback-on-failure is explicitly requested
  • existing resource groups are not recreated and their customer tags are
    preserved
  • healthy, complete existing AKS deployments are reused without applying the
    full managed-cluster Bicep template
  • existing Kubernetes versions, pool names, VM SKUs, pool counts, support
    plans, and upgrade policies are not silently changed
  • existing pool/version mutations fail before Azure writes and direct operators
    to supported AKS workflows
  • legacy deployment contexts use unique subscription discovery and fail closed
    on missing or ambiguous targets

Validation

  • 68 CLI test files passed: 1,156 passed, 2 skipped
  • TypeScript typecheck and package build passed
  • oxlint completed with 0 errors; remaining warnings predate this change
  • Bicep CLI 0.44.1 build and lint passed
  • generated and npm-bundled assets contain no trustPolicy or hardcoded 1.33
  • live read-only West US 3 checks selected AKS 1.36.3 and calculated the
    expected 8-vCPU one-sandbox footprint
  • existing resource-group tags remained byte-for-byte unchanged
  • retained ACR, Key Vault, managed identity, and external Foundry resources
    resolved successfully
  • purge-protected Key Vault discovery and recovery passed against a real
    soft-deleted vault
  • the final draft head has no failed or pending CI checks, including LOC,
    security-audit, E2E, chaos, benchmark, Bicep, dependency, secret, CodeQL,
    Trivy, and container gates

Draft blocker

The disposable full deployment passed account, naming, quota, provider, and ARM
validation and reached AKS/RBAC creation. It then failed because the available
subscriptions deny:

Microsoft.Authorization/roleAssignments/write

All disposable resource groups were deleted. A purge-protected test Key Vault
tombstone remains recoverable until its scheduled purge.

Before marking this PR ready:

  • rerun the full disposable customer deployment in a subscription granting
    Microsoft.Authorization/roleAssignments/write
  • confirm generated-resource-group rollback and management-lock cleanup live
  • ask the reporter to retest the original kars up path

Fixes #515
Fixes #516
Fixes #517
Fixes #518

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: eb3654cd-f1e0-445a-8734-430800af1903
@github-actions

github-actions Bot commented Sep 2, 2026

Copy link
Copy Markdown

Dependency Review

✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.

Scanned Files

None

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: eb3654cd-f1e0-445a-8734-430800af1903
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT License.

import { describe, expect, it, vi } from "vitest";
];
};

const azureDataWithUnavailableSystem = (
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT License.

import { describe, expect, it, vi } from "vitest";
];
};

const azureDataWithUnavailableSystem = (
@pallakatos

Copy link
Copy Markdown
Collaborator Author

The full CI/security matrix is now green on ecca448b. Before moving this draft toward merge, we would appreciate reporter validation against the original Azure failures.

Jess (@appliedi), if you have a disposable non-production subscription with Microsoft.Authorization/roleAssignments/write, you can run the branch CLI without installing it globally:

git clone https://github.com/Azure/kars.git
cd kars
git fetch origin pull/531/head:pr-531
git switch pr-531
cd cli
npm ci
npm run build
node dist/index.js --version

For a combined fresh-deployment check in West US 3:

node dist/index.js up \
  --name deployment-safety-test \
  --region westus3 \
  --release v0.1.26 \
  --rollback-on-failure \
  --yes

Please use only disposable resources. The CLI prints the generated resource group, subscription, selected Kubernetes version, computed node footprint, and rollback lease. If the deployment succeeds, remove it afterward with the exact generated values:

node dist/index.js destroy --all --yes \
  --resource-group <generated-rg> \
  --subscription <subscription-id> \
  --region westus3

Please report the selected AKS version/node count, whether ACR and RBAC deployment completed, whether rollback/cleanup completed on any failure, and any unexpected behavior here on #531. Redact subscription IDs, tenant IDs, resource names, and credentials from logs. Findings on any of #515#518 can be centralized in this PR so we can fix them before merge.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: eb3654cd-f1e0-445a-8734-430800af1903
Preserve custom node resource groups for existing AKS clusters while retaining fail-fast validation for dry runs and newly created clusters.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: ee78c86b-0001-4d24-b829-7c75b59c9316
@pallakatos

Copy link
Copy Markdown
Collaborator Author

CI is now fully green on e353d98f with no failed or pending checks. The custom node-resource-group follow-up briefly regressed three preflight tests; the final fix preserves existing custom AKS node resource groups while retaining fail-fast generated-name validation for dry runs and genuinely new clusters.

Jess (@appliedi), the branch-testing instructions above are now unpaused and refer to the current PR head. Please fetch pull/531/head again before testing so you receive e353d98f.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: eb3654cd-f1e0-445a-8734-430800af1903
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: eb3654cd-f1e0-445a-8734-430800af1903
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants