docs(spec): plan PD roles and cross-role atomic admission for ModelDeployment

[thxCode]

What type of PR is this?

/kind documentation
/area worker

What this PR does / why we need it:

Adds the design document for cross-role PD (prefill/decode) workloads and Kueue atomic
admission. Spec only — 1 file, no implementation code, following the four docs(spec):
precedents on main (739d7b5a, 8c2cc0df, 674f5583, 57bfd123).

Prefill and decode only work if both halves run: a prefiller with no decoder produces KV blocks
nobody consumes, and a decoder with no prefiller has nothing to decode. The single-role
ModelDeployment spec creates one Kueue Workload per replica — correct for replicas that are
independently useful, and exactly wrong here: on a pool that fits half of them, Kueue admits half,
and which half is a race.

The spec lifts spec.roles from "exactly 1" to 1..10 and makes a deployment's Pods one Kueue
pod group, so every role is admitted all-or-nothing. No new CRD. Three per-role fields:
kind (what the engine is told it is), acceleratorKey (which accelerator model inside the pool
the role wants), and nothing else.

Two constraints shape the whole design, both verified in this tree:

• A Workload carries one queueName, so every role must sit in one ClusterQueue and
  roles[*].instanceType must be identical. The spec does not relax that and cannot.
• Kueue assigns a ResourceFlavor per PodSet (flavorassigner.go: each PodSet forms its own
  assignment group by default), so inside one ClusterQueue two roles can still land on two
  accelerator models of the same manufacturer.

Across manufacturers stays impossible, structurally rather than for want of a feature: an
accelerated queue's quota is credits.gpustack.ai/<manufacturer> — one resource name per
manufacturer — and Kueue rejects a second resource group repeating a covered resource.

Status: Specified. The plan states one exception to its own ordering: the per-role feasibility
fix (F6/T10) ships first, on its own, because it depends on neither the single-role spec nor
cross-model pools and repairs a defect reachable today.

Which issue(s) this PR links to:

None — this spec was not filed as an issue.

Special notes for your reviewer:

• The spec is self-contained by design. Every external conclusion it uses is quoted with its
  evidence in the body, and external sources are cited by URL; it references no local file.
• Verification honesty is explicit, not implied. Heterogeneous placement needs two accelerator
  models of one manufacturer in one pool, which the available hardware cannot provide, so it is
  verified in envtest against two fake ResourceFlavors — and the spec says why the fake is
  faithful at that layer: the property asserted ("Kueue assigned two different flavors to the two
  PodSets") is a Workload-status property, not a hardware one. The local-cluster case verifies
  atomic admission by starvation, asserting a negative (zero Pods of either role running).
  No throughput, latency or RDMA claim is made anywhere.
• Two findings about existing code are recorded in Notes, and neither is in any task's Owns
  here:
  1. NodeDevicesAdmissionReconciler is not role-aware — that is F6/T10, shipping as its own PR.
  2. buildResourceGroups (node_queue.go:280) reads Kueue's MaxItems=16 as "16 flavors per
     resource group" and opens a second group with the same CoveredResources past 16. Kueue caps
     resource groups at 16 and flavors per group at 64, and validateResourceGroups
     declares its seenResources set outside the per-group loop, so a repeated covered resource is
     a field.Duplicate and the ClusterQueue update is rejected. A single-covered-resource queue
     can therefore only ever have one group; the split is itself the bug and the real ceiling
     is 64.
• The deployed Kueue is 0.18.4 while the Go module is v0.17.1 (hack/deps.sh vs go.mod).
  Every runtime behaviour the spec relies on was read from 0.18.4, the version that runs; the
  skew is recorded as a standing caveat so the next reader does not take go.mod for the truth.

Does this PR introduce a user-facing change?

NONE

[gpustack-codereview]

✅ OpenCodeReview: Review skipped: no items were selected.

[thxCode]

Closing this: opening it was premature, and the repository's own history says so more clearly than
the precedent I opened it on.

Three of the four docs(spec): commits I cited did land before their implementation — but they
landed as Status: Planned, and their work started immediately after. Across all 33 specs on
main, the statuses ever used on first landing are Shipped (21), Building (8), Planned (3)
and Built (1). Specified — what this spec carries — has never appeared on main; it is the
drafting state.

Two things make this case unlike those three precedents:

• This spec's implementation is blocked on the single-role ModelDeployment spec, which has not
  started. So the gap between landing and building is open-ended rather than days.
• It specifies 10 features and one is built. The design has not yet been validated by
  implementation, so an amendment would land on main.

The spec is unchanged on the kvc-s7-spec branch and will land when the work starts — as
Building, or with the implementation.

#159 is unaffected and stays open. It repairs a defect reachable on main today (a node
carrying two accelerator models has its other model's cards counted toward a demand its flavor does
not cover), carries its own evidence, and does not depend on this spec being merged. Its
description has been updated so it no longer refers to this PR.

[Copilot]

🟡 Changes recommended

Critical and moderate correctness gaps remain in Service isolation, role composition, validation, status, naming, and testing.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Adds a design specification for multi-role ModelDeployment workloads with atomic Kueue admission.

Changes:

• Defines role APIs, validation, placement, Services, and status.
• Plans per-role flavor selection and feasibility checks.
• Outlines implementation and testing.

File summaries

File	Description
specs/2026-09-02-model-deployment-pd-atomic-admission.md	Designs cross-role atomic admission and implementation milestones.

Review details

Suppressed comments (2)

specs/2026-09-02-model-deployment-pd-atomic-admission.md:16

• The summary says three fields but names only two; the implementation plan confirms two additions to ModelDeploymentRole (kind and acceleratorKey) plus separate status fields. State that this adds two per-role spec fields so the public API summary is unambiguous.

length 1; this spec lifts that bound to 1..10 and adds three per-role fields — `kind` (what the
engine is told it is), `acceleratorKey` (which accelerator model inside the pool this role wants) and
nothing else. The Pods of a deployment become one **Kueue pod group**, one PodSet per role, and the

specs/2026-09-02-model-deployment-pd-atomic-admission.md:1006

• This API example contradicts F8 and T1: ModelDeploymentRoleStatus is required to gain both kind and assignedFlavor, but the shown struct adds only AssignedFlavor. Add Kind to the example and reconcile the summary's per-role field count so T1 has one unambiguous API shape.

type ModelDeploymentRoleStatus struct {

• Files reviewed: 1/1 changed files
• Comments generated: 9
• Review effort level: Balanced