Skip to content

feat(examples): add gpu-k8s-arc-sglang GPU Kubernetes deployment example - #422

Draft
powderluv wants to merge 1 commit into
mainfrom
users/powderluv/gpu-k8s-arc-sglang-example
Draft

feat(examples): add gpu-k8s-arc-sglang GPU Kubernetes deployment example#422
powderluv wants to merge 1 commit into
mainfrom
users/powderluv/gpu-k8s-arc-sglang-example

Conversation

@powderluv

Copy link
Copy Markdown
Collaborator

Summary

Adds a self-contained example under examples/gpu-k8s-arc-sglang/ for standing up a native k0s GPU Kubernetes cluster on SPUR nodes: ARC (actions-runner-controller) CI runners for a sglang fork, gated auto-deploy of an sglang serving Deployment, and optional backend RoCE/RDMA for distributed inference.

Genericized from a working AMD MI355X (gfx950) 4-node deployment.

What's included

  • Cluster (10-cluster/) — k0s + Calico bird native routing over a WireGuard mesh (single overlay).
  • GPU (20-gpu/) — AMD GPU device plugin + node labelling.
  • ARC (30-arc/) — GPU runner scale set (hand-authored dind spec so /dev/kfd+/dev/dri reach the dind sidecar) and a non-GPU deploy runner with least-privilege RBAC; GitHub App setup docs.
  • Serving (40-serving/) — sglang Deployment/Service/PVC/Ingress (ROCm, ungated model) + a gated fork-CI deploy job that kubectl applys on green CI via an in-cluster ServiceAccount (scoped to the sglang namespace).
  • RDMA/DI (50-rdma/) — RoCEv2 pod exposure (hostNetwork + SR-IOV options), 2-node tensor-parallel, and prefill/decode disaggregation (MoRI) reference manifests.
  • Top-level bootstrap.sh / teardown.sh / Makefile / README.md / RUNBOOK.md template.

Genericization / safety

Redacted for public publication: hostnames → gpu-node-N / head-node, provider mount path → /mnt/scratch, the internal operational RUNBOOK replaced with a fill-in template. Mesh/rail/pod subnets (10.44.0.0/16, etc.) are RFC1918 example config the example itself provisions. No secrets/tokens/keys — github-app-secret.example.yaml is a placeholder template.

Status

Opened as a draft / review branch. Cluster bring-up, the GPU device plugin, ARC runners, and the gated serving deploy path were validated on real MI355X hardware. The 50-rdma/ distributed-inference manifests are references — cross-node RDMA KV-transfer has an open NIC-QoS (nicctl PFC/DCQCN) dependency noted inline.

🤖 Generated with Claude Code

A self-contained example for a native k0s GPU Kubernetes cluster on SPUR
nodes: ARC (actions-runner-controller) CI runners for a sglang fork, gated
auto-deploy of an sglang serving Deployment, and optional backend RoCE/RDMA
for distributed inference.

Genericized from a working AMD MI355X (gfx950) deployment: hostnames ->
gpu-node-N, provider mount path -> /mnt/scratch, RUNBOOK.md is a fill-in
template. Mesh/rail/pod subnets are RFC1918 example config.

Changes:
- examples/gpu-k8s-arc-sglang/{00-network..50-rdma} staged deploy tree
- k0s + Calico bird native routing over a WireGuard mesh; AMD GPU device plugin
- ARC GPU runner scale set (hand-authored dind spec) + non-GPU deploy runner
  with scoped RBAC; gated fork-CI deploy job
- sglang serving Deployment/Service/PVC/Ingress (ROCm, ungated model)
- 50-rdma: RoCEv2 pod exposure + 2-node TP and prefill/decode disagg examples
- bootstrap.sh / teardown.sh / Makefile / README / RUNBOOK template

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@codecov-commenter

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #422      +/-   ##
==========================================
- Coverage   70.86%   70.85%   -0.01%     
==========================================
  Files         140      140              
  Lines       41849    41849              
==========================================
- Hits        29656    29652       -4     
- Misses      12193    12197       +4     
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants