Skip to content

Kubernetes: workload provisioning #461

Description

@ArshSSandhu

Parent: #441

Depends on: Kubernetes API client and cluster discovery

Summary

Allow Manager to provision a Manager Container as a Kubernetes Deployment.

A Manager Container should map to a stable Kubernetes Deployment rather than directly to a Pod because Pods may be replaced during scheduling, failures, or rollouts.

Resource mapping

Manager Kubernetes
Node Kubernetes cluster/context
Container Deployment
Runtime instance Pod
Provider ID namespace/deployment-name

Scope

Implement Kubernetes workload creation through KubernetesApi.

Create an apps/v1 Deployment with:

  • one replica
  • OCI image
  • Manager hostname/workload name
  • CPU configuration
  • memory configuration
  • environment variables
  • entrypoint/command
  • Manager ownership labels
  • configured namespace

Ownership labels

Manager-created resources should include namespaced labels such as:

org.mieweb.opensource-server.managed=true

org.mieweb.opensource-server.node-id=<node-id>

org.mieweb.opensource-server.container-id=<container-id>

Apply appropriate labels to both the Deployment and its Pod template.

Provider ID

Persist a stable workload identifier in Container.containerId.

Use:

namespace/deployment-name

Do not permanently store:

  • Pod name
  • Pod UID
  • ReplicaSet name

Image handling

Kubernetes should receive the OCI image reference directly.

Do not execute Proxmox image-storage or OCI archive logic for Kubernetes workloads.

Kubernetes/container runtime should handle image pulling and caching.

Resource mapping

Map Manager resources to Kubernetes quantities.

Examples:

  • 1 CPU → 1000m
  • 2 CPUs → 2000m
  • 1024 MB1024Mi

Readiness

After creating the Deployment, wait for it to become available.

Use a bounded timeout.

Creation should succeed when the requested replica is available and the Deployment rollout has completed.

Shared create flow

Refactor the existing creation path as needed so Kubernetes does not require:

  • Proxmox VMID allocation
  • Proxmox storage resolution
  • LXC cloning
  • Proxmox ACL setup
  • Proxmox hook scripts
  • stable MAC discovery

Avoid spreading nodeType === 'kubernetes' checks throughout unrelated code.

Acceptance criteria

  • Manager can create a Kubernetes Deployment.
  • Manager Container maps to a Deployment.
  • Deployment starts with one replica.
  • OCI image is passed directly to Kubernetes.
  • CPU is mapped correctly.
  • Memory is mapped correctly.
  • Environment variables are applied.
  • Entrypoint/command is applied.
  • Manager ownership labels are added.
  • Stable Deployment provider ID is stored.
  • Manager waits for Deployment readiness.
  • Creation has a bounded timeout.
  • Kubernetes creation does not execute Proxmox storage/VMID paths.
  • Unit tests cover manifest generation and creation.
  • Existing providers continue to work.

Out of scope

  • Start/stop
  • Delete
  • Live status
  • Reconfiguration
  • Services
  • Ingress
  • Persistent volumes

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions