Skip to content

feat(helm): support existing Kubernetes clusters - #543

Open
Pal Lakatos-Toth (pallakatos) wants to merge 10 commits into
mainfrom
public/pr1-generic-kubernetes
Open

feat(helm): support existing Kubernetes clusters#543
Pal Lakatos-Toth (pallakatos) wants to merge 10 commits into
mainfrom
public/pr1-generic-kubernetes

Conversation

@pallakatos

@pallakatos Pal Lakatos-Toth (pallakatos) commented Sep 3, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • add an opt-in values-generic.yaml profile for existing non-AKS Kubernetes clusters
  • deploy the published Microsoft AGT AgentMesh relay and registry for the generic Helm profile
  • make sandbox node selection configurable while preserving the existing AKS pool selector by default
  • emit Azure Workload Identity metadata only when the existing value is enabled
  • render default, local-kind, and generic profiles in CI
  • document Helm installation and CRD ownership without requiring the Kars CLI

Images

The generic profile uses the image names produced by the public release workflow:

  • ghcr.io/azure/kars-controller:latest
  • ghcr.io/azure/kars-inference-router:latest
  • ghcr.io/azure/openclaw-sandbox:latest
  • ghcr.io/azure/kars-agentmesh-relay:latest
  • ghcr.io/azure/kars-agentmesh-registry:latest

All five published manifests were verified directly.

AgentMesh ownership

agentMesh.enabled defaults to false so existing kars up and kars dev flows continue managing AgentMesh separately without duplicate Helm ownership. values-generic.yaml enables it, producing the agentmesh namespace, relay/registry Deployments, and the stable agentmesh-relay:8765 / agentmesh-registry:8080 Services expected by Kars routers and runtimes. Operators with an external AgentMesh deployment can set agentMesh.enabled=false.

CRDs

The chart renders and installs every Kars CustomResourceDefinition included in that chart version. The current public base renders 12 CRDs; later stacked API PRs add their CRD templates to the same Helm lifecycle.

Backward compatibility

The default values remain AKS-first. sandbox.nodeSelector defaults to {}, which resolves to the existing kars.azure.com/pool=<isolation pool> selector. azure.workloadIdentity.enabled remains true, so existing AKS installs retain current identity metadata. AgentMesh remains disabled in default values because existing CLI flows already deploy it.

Validation

  • cargo test --package kars-controller: 860 passed
  • phase taxonomy guard: passed
  • CNCF conformance criteria: 17/17 passed, including the new AgentMesh Deployments
  • cargo clippy --all-targets --all-features -- -D warnings: passed
  • cargo fmt --all -- --check: passed
  • CLI release-asset tests and typecheck: passed
  • Helm lint plus default/local-kind/generic rendering: passed
  • all five GHCR image manifests verified
  • LOC gate: passed

This is PR 1 of the ordered Kars-Bridge publicization stack and contains no Bridge orchestration, inference, or runtime API changes.

Add an opt-in generic Kubernetes values profile, portable sandbox node selection, conditional Azure Workload Identity metadata, Helm render coverage, and installation guidance without changing AKS defaults.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: eb3654cd-f1e0-445a-8734-430800af1903
@github-actions

github-actions Bot commented Sep 3, 2026

Copy link
Copy Markdown

Dependency Review

✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.

Scanned Files

None

Deploy the published Microsoft AGT relay and registry from the generic Helm profile while keeping the component disabled by default for existing CLI and AKS flows that already manage AgentMesh separately.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: eb3654cd-f1e0-445a-8734-430800af1903
@pallakatos
Pal Lakatos-Toth (pallakatos) marked this pull request as draft September 4, 2026 06:59
@pallakatos
Pal Lakatos-Toth (pallakatos) marked this pull request as ready for review September 4, 2026 07:53
@pallakatos

Copy link
Copy Markdown
Collaborator Author

Follow-up from review of the Helm contract:

  • Verified all five configured public manifests exist: kars-controller, kars-inference-router, openclaw-sandbox, kars-agentmesh-relay, and kars-agentmesh-registry under ghcr.io/azure, all at :latest as required by Kars image-tag policy.
  • values-generic.yaml now enables agentMesh, so a generic Helm install includes the AGT relay/registry and stable Services required for inter-agent communication.
  • Default values keep agentMesh.enabled=false to avoid double ownership in existing kars up / kars dev flows that already deploy AgentMesh separately.
  • Helm renders and installs every CRD in that chart version; the current feat(helm): support existing Kubernetes clusters #543 base contains 12 CRDs, while the stacked governance/team PRs add their new CRD templates.
  • Revised head a12aa63e is fully CI-green after rerunning an external npm audit endpoint failure.

Verify an existing Kars Helm release and CRD, then persist the explicit AKS, subscription, and ACR metadata required by upgrade, push, mesh, and advanced add flows without mutating infrastructure.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: eb3654cd-f1e0-445a-8734-430800af1903
@pallakatos
Pal Lakatos-Toth (pallakatos) marked this pull request as draft September 4, 2026 08:42
@pallakatos

Copy link
Copy Markdown
Collaborator Author

Existing-AKS / kars up correlation completed:

  • Kubernetes-facing commands (connect, list, operator, destroy <sandbox>, policy/CRD commands, etc.) already recognize any explicit/current kube context and do not require kars up provenance.
  • Azure lifecycle commands (upgrade, push, mesh promote/demote, and advanced add recovery/provisioning) depend on ~/.kars/context.json, which previously only kars up wrote.
  • Added kars config adopt-aks on head c9ec1edd. It verifies the selected cluster contains the Kars Helm release and KarsSandbox CRD, validates the ACR host, and writes the explicit subscription/region/RG/cluster/ACR metadata without changing Azure or Kubernetes infrastructure. Optional WI/OIDC/identity/Foundry/Key Vault flags enable the corresponding advanced flows.
  • Added focused tests, docs, and a security audit. The PR is back in draft while CI reruns.

Allow cluster-aware commands to proceed with an explicit --context even when kubeconfig has no global current-context, preserving safe targeting for Helm-installed clusters.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: eb3654cd-f1e0-445a-8734-430800af1903
@pallakatos

Copy link
Copy Markdown
Collaborator Author

kars operator correlation:

  • The operator is Kubernetes-native and does not require ~/.kars/context.json; it discovers Kars CRDs/pods from --context, KARS_KUBE_CONTEXT, the current context, or all reachable kube contexts.
  • Therefore a Helm-installed AKS cluster appears in kars operator as soon as its kube context is available and the caller has RBAC to list Kars resources.
  • Fixed an edge case where the global bootstrap rejected kars operator --context <name> if no global current-context existed. Head 58c1cf68 now treats the explicit flag as the authoritative safe selection; 36 focused config/bootstrap tests pass.
  • kars config adopt-aks remains necessary only for Azure lifecycle commands that need subscription/RG/ACR metadata, not for the operator dashboard itself.

Document Helm-managed AgentMesh and CRDs, existing AKS adoption, operator kube-context discovery, and the distinction between Kubernetes-facing and Azure lifecycle commands.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: eb3654cd-f1e0-445a-8734-430800af1903
@pallakatos

Copy link
Copy Markdown
Collaborator Author

Documentation is now updated on head 7fc0dbc7 across the public discovery surfaces:

  • top-level README.md: kars up versus existing-cluster Helm path + adoption example
  • docs/README.md: direct existing-cluster installation entry point
  • docs/getting-started.md: corrected current Helm value model, AgentMesh ownership, CLI adoption, and operator behavior
  • docs/how-to/helm-installation.md: generic Kubernetes and existing AKS procedures
  • docs/cli-reference.md: full kars config adopt-aks reference and the Kubernetes-vs-Azure command distinction
  • chart-local deploy/helm/kars/README.md: AgentMesh, CRD, and adoption ownership contract

The PR remains draft while CI runs on the latest documentation/code head.

Provide a render-tested values file mirroring the Helm contract emitted by kars up, including images, Foundry, identity, federated credentials, Key Vault, AgentMesh, and release metadata.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: eb3654cd-f1e0-445a-8734-430800af1903
Require an exact common release stamp and document ACR image import, AcrPull, Workload Identity, Foundry, Key Vault, networking, and node prerequisites for a successful manual AKS install.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: eb3654cd-f1e0-445a-8734-430800af1903
@pallakatos

Copy link
Copy Markdown
Collaborator Author

Added a checked-in, render-tested deploy/helm/kars/values-existing-aks.yaml template derived directly from the Helm flags emitted by kars up.

It covers:

  • controller, router, sandbox, and all runtime image repositories/tags
  • Azure OpenAI + Foundry account/project/deployment catalog
  • Content Safety
  • controller Workload Identity and kubelet IMDS identity
  • Key Vault CSI
  • federated-credential subscription/identity/OIDC metadata
  • AgentMesh ownership
  • portable initial sandbox scheduling
  • exact karsRelease stamp

The docs now require importing the referenced images into ACR, granting AKS kubelet AcrPull, using one exact release tag across components, and pre-provisioning the Azure identity/RBAC resources. CI renders this template alongside default/local/generic profiles.

Make the existing-AKS template pull verified ghcr.io/azure images by default and make ACR metadata optional during CLI adoption unless private mirroring, push, or ACR-based upgrade is required.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: eb3654cd-f1e0-445a-8734-430800af1903
@pallakatos

Copy link
Copy Markdown
Collaborator Author

Corrected the existing-AKS template on head 5b50caff: Kars image repositories are now fixed to the verified public ghcr.io/azure/* locations. Customer ACR is optional, used only for private mirroring or the current ACR-based kars push / kars upgrade flows. kars config adopt-aks now accepts installations without an ACR mirror. Focused CLI tests (34) and Helm rendering passed; CI is rerunning.

Keep high-severity npm audit blocking while avoiding invalid installed-tree errors from the OpenClaw local file dependency.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: eb3654cd-f1e0-445a-8734-430800af1903
Audit dependency lockfiles through npm's supported bulk endpoint with bounded retries and fail-closed high/critical handling, replacing the retired quick-audit endpoint.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: eb3654cd-f1e0-445a-8734-430800af1903
@pallakatos
Pal Lakatos-Toth (pallakatos) marked this pull request as ready for review September 4, 2026 11:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant