diff --git a/docs/developers/scheduling.md b/docs/developers/scheduling.md index 4597384e0..2c7550a74 100644 --- a/docs/developers/scheduling.md +++ b/docs/developers/scheduling.md @@ -94,15 +94,15 @@ score: ((request + used) / allocatable) * 10 1. Binpack scoring information for Node 1 is as follows -```text -Node1 score: ((1+3)/4) * 10= 10 -``` + ```text + Node1 score: ((1+3)/4) * 10= 10 + ``` 1. Binpack scoring information for Node 2 is as follows -```text -Node2 score: ((1+2)/4) * 10= 7.5 -``` + ```text + Node2 score: ((1+2)/4) * 10= 7.5 + ``` In `Binpack` policy, `Node1` is selected. @@ -116,15 +116,15 @@ score: ((request + used) / allocatable) * 10 1. Spread scoring information for Node 1 is as follows -```text -Node1 score: ((1+3)/4) * 10= 10 -``` + ```text + Node1 score: ((1+3)/4) * 10= 10 + ``` 1. Spread scoring information for Node 2 is as follows -```text -Node2 score: ((1+2)/4) * 10= 7.5 -``` + ```text + Node2 score: ((1+2)/4) * 10= 7.5 + ``` In `Spread` policy, `Node2` is selected. @@ -132,47 +132,96 @@ In `Spread` policy, `Node2` is selected. ![HAMi GPU scheduler policy diagram, comparing Binpack and Spread scores on each card](/img/docs/common/developers/scheduling/gpu-scheduler-policy-demo.png) -#### Binpack +#### Per-Pod device scoring weights -Binpack mainly focuses on the computing power and video memory usage of each card. The more it is used, the higher the score. +By default, HAMi gives equal influence to predicted virtual-device slot, device-core, and device-memory utilization when it scores a physical device. To change that balance for one workload, add the `hami.io/device-scoring-weights` annotation to the Pod: -```text -score: ((request.core + used.core) / allocatable.core + (request.mem + used.mem) / allocatable.mem)) * 10 +```yaml +apiVersion: v1 +kind: Pod +metadata: + name: memory-weighted-gpu-pod + annotations: + hami.io/device-scoring-weights: "slot=1,core=1,memory=3" +spec: + containers: + - name: workload + image: ubuntu:22.04 + command: ["bash", "-c", "sleep 86400"] + resources: + limits: + nvidia.com/gpu: 1 + nvidia.com/gpumem-percentage: 40 ``` -1. Binpack scoring information for GPU 1 is as follows +HAMi predicts each candidate device's utilization after placing the request, then calculates its device score as follows: ```text -GPU1 Score: ((20+10)/100 + (1000+2000)/8000)) * 10 = 6.75 +score = 10 * ( + slotWeight * predictedSlotUtilization + + coreWeight * predictedCoreUtilization + + memoryWeight * predictedMemoryUtilization +) ``` -1. Binpack scoring information for GPU 2 is as follows +The annotation must contain the `slot`, `core`, and `memory` keys. Each value must be a non-negative integer, and at least one value must be greater than zero. Key order and surrounding whitespace do not matter. If the annotation is absent, HAMi uses `slot=1,core=1,memory=1`, which preserves the default scoring behavior. When the admission webhook is enabled, invalid annotations are rejected when a Pod requesting a HAMi-managed resource is created. If admission validation is unavailable, the scheduler rejects the scheduling attempt and the Pod remains unschedulable until the annotation is corrected. + +For example, consider two candidate GPUs after accounting for a Pod that requests one vGPU and 40% device memory: + +| Device | Predicted slot utilization | Predicted core utilization | Predicted memory utilization | +| ------ | -------------------------: | -------------------------: | ---------------------------: | +| GPU A | 0.2 | 0.9 | 0.5 | +| GPU B | 0.8 | 0.1 | 0.6 | + +With the default `1:1:1` weights, GPU A scores `16` and GPU B scores `15`, so `binpack` prefers GPU A. With `slot=1,core=1,memory=3`, GPU A scores `26` and GPU B scores `27`, so `binpack` prefers GPU B. Under `spread`, the lower score is preferred instead. + +The annotation changes only the utilization score used to order candidate devices. It does not bypass device fit or capacity checks, mutex rules, NUMA or topology constraints, or vendor-specific `Fit` behavior. These constraints keep their existing precedence; when topology candidates are otherwise tied, their utilization-score ordering can act as the tie-breaker. + +#### Binpack + +Binpack prefers the card with the higher device-utilization score. The following default-weight example assumes each card has ten virtual-device slots and exactly one slot is currently in use: ```text -GPU2 Score: ((20+70)/100 + (1000+6000)/8000)) * 10 = 17.75 +score: ((request.slot + used.slot) / allocatable.slot + + (request.core + used.core) / allocatable.core + + (request.mem + used.mem) / allocatable.mem) * 10 ``` +1. Binpack scoring information for GPU 1 is as follows + + ```text + GPU1 Score: ((1+1)/10 + (20+10)/100 + (1000+2000)/8000) * 10 = 8.75 + ``` + +1. Binpack scoring information for GPU 2 is as follows + + ```text + GPU2 Score: ((1+1)/10 + (20+70)/100 + (1000+6000)/8000) * 10 = 19.75 + ``` + In `Binpack` policy, `GPU2` is selected. #### Spread -Spread mainly focuses on the computing power and video memory usage of each card. The less it is used, the higher the score. +Spread prefers the card with the lower device-utilization score. Using the same default-weight example: ```text -score: ((request.core + used.core) / allocatable.core + (request.mem + used.mem) / allocatable.mem)) * 10 +score: ((request.slot + used.slot) / allocatable.slot + + (request.core + used.core) / allocatable.core + + (request.mem + used.mem) / allocatable.mem) * 10 ``` 1. Spread scoring information for GPU 1 is as follows -```text -GPU1 Score: ((20+10)/100 + (1000+2000)/8000)) * 10 = 6.75 -``` + ```text + GPU1 Score: ((1+1)/10 + (20+10)/100 + (1000+2000)/8000) * 10 = 8.75 + ``` 1. Spread scoring information for GPU 2 is as follows -```text -GPU2 Score: ((20+70)/100 + (1000+6000)/8000)) * 10 = 17.75 -``` + ```text + GPU2 Score: ((1+1)/10 + (20+70)/100 + (1000+6000)/8000) * 10 = 19.75 + ``` In `Spread` policy, `GPU1` is selected. diff --git a/docs/userguide/configure.md b/docs/userguide/configure.md index e3ec92e6f..681f8c85b 100644 --- a/docs/userguide/configure.md +++ b/docs/userguide/configure.md @@ -81,6 +81,7 @@ helm install hami hami-charts/hami --set devicePlugin.deviceMemoryScaling=5 -n k | `nvidia.com/use-gputype` | String | If set, devices allocated by this pod MUST be one of the types defined in this string. | `"Tesla V100-PCIE-32GB, NVIDIA A10"` | | `hami.io/node-scheduler-policy` | String | GPU node scheduling policy: `"binpack"` allocates the pod to used GPU nodes for execution. `"spread"` allocates the pod to different GPU nodes for execution. | `"binpack"` or `"spread"` | | `hami.io/gpu-scheduler-policy` | String | GPU scheduling policy: `"binpack"` allocates the pod to the same GPU card for execution. `"spread"` allocates the pod to different GPU cards for execution. `"mutex"` allocates the pod only to a GPU card with no other workloads, giving it exclusive use of that card. | `"binpack"`, `"spread"` or `"mutex"` | +| `hami.io/device-scoring-weights` | String | Relative weights of virtual-device slot, device-core, and device-memory utilization in physical-device scoring. All three weights are required, must be non-negative integers, and at least one must be positive. | `"slot=1,core=1,memory=3"` | | `nvidia.com/vgpu-mode` | String | The type of vGPU instance this pod wishes to use. | `"hami-core"` or `"mig"` | ## Container Configs: Env diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/developers/scheduling.md b/i18n/zh/docusaurus-plugin-content-docs/current/developers/scheduling.md index 8beb35c06..3facd2212 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/developers/scheduling.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/developers/scheduling.md @@ -95,15 +95,15 @@ score: ((request + used) / allocatable) * 10 1. 节点 1 的 Binpack 评分信息如下 -```text -Node1 score: ((1+3)/4) * 10= 10 -``` + ```text + Node1 score: ((1+3)/4) * 10= 10 + ``` 1. 节点 2 的 Binpack 评分信息如下 -```text -Node2 score: ((1+2)/4) * 10= 7.5 -``` + ```text + Node2 score: ((1+2)/4) * 10= 7.5 + ``` 因此，在 `Binpack` 策略中我们可以选择 `Node1`。 @@ -117,15 +117,15 @@ score: ((request + used) / allocatable) * 10 1. 节点 1 的 Spread 评分信息如下 -```text -Node1 score: ((1+3)/4) * 10= 10 -``` + ```text + Node1 score: ((1+3)/4) * 10= 10 + ``` 1. 节点 2 的 Spread 评分信息如下 -```text -Node2 score: ((1+2)/4) * 10= 7.5 -``` + ```text + Node2 score: ((1+2)/4) * 10= 7.5 + ``` 因此，在 `Spread` 策略中我们可以选择 `Node2`。 @@ -133,46 +133,95 @@ Node2 score: ((1+2)/4) * 10= 7.5 ![HAMi GPU 调度策略示意图，展示在单卡上的 Binpack 与 Spread 评分对比](/img/docs/common/developers/scheduling/gpu-scheduler-policy-demo.png) -#### Binpack +#### 每个 Pod 的设备评分权重 + +默认情况下，HAMi 在计算物理设备得分时，会让预测的虚拟设备槽位、设备核心和设备显存利用率具有相同的影响。若要为某个工作负载调整它们的相对影响，可为 Pod 添加 `hami.io/device-scoring-weights` 注解： + +```yaml +apiVersion: v1 +kind: Pod +metadata: + name: memory-weighted-gpu-pod + annotations: + hami.io/device-scoring-weights: "slot=1,core=1,memory=3" +spec: + containers: + - name: workload + image: ubuntu:22.04 + command: ["bash", "-c", "sleep 86400"] + resources: + limits: + nvidia.com/gpu: 1 + nvidia.com/gpumem-percentage: 40 +``` -Binpack 主要关注每张卡的计算能力和显存使用情况。使用越多，得分越高。 +HAMi 会预测在放置请求后每个候选设备的利用率，然后按以下公式计算设备得分： ```text -score: ((request.core + used.core) / allocatable.core + (request.mem + used.mem) / allocatable.mem)) * 10 +score = 10 * ( + slotWeight * predictedSlotUtilization + + coreWeight * predictedCoreUtilization + + memoryWeight * predictedMemoryUtilization +) ``` -1. GPU1 的 Binpack 评分信息如下 +注解必须包含 `slot`、`core` 和 `memory` 三个键。每个值都必须是非负整数，并且至少有一个值大于零。键的顺序以及两侧的空格不会影响解析。如果未设置该注解，HAMi 将使用 `slot=1,core=1,memory=1`，从而保持默认评分行为。启用准入 Webhook 后，如果请求 HAMi 管理资源的 Pod 包含无效注解，则会在创建时被拒绝。如果准入验证不可用，调度器会拒绝此次调度尝试，Pod 将保持不可调度状态，直到该注解被修正。 + +例如，假设一个 Pod 请求 1 个 vGPU 和 40% 的设备显存，并且在计入该请求后有两个候选 GPU： + +| 设备 | 预测槽位利用率 | 预测核心利用率 | 预测显存利用率 | +| ----- | -------------: | -------------: | -------------: | +| GPU A | 0.2 | 0.9 | 0.5 | +| GPU B | 0.8 | 0.1 | 0.6 | + +使用默认的 `1:1:1` 权重时，GPU A 得分为 `16`，GPU B 得分为 `15`，因此 `binpack` 会优先选择 GPU A。使用 `slot=1,core=1,memory=3` 时，GPU A 得分为 `26`，GPU B 得分为 `27`，因此 `binpack` 会优先选择 GPU B。在 `spread` 策略下，则会优先选择得分较低的设备。 + +该注解只会改变用于排序候选设备的利用率得分。它不会绕过设备适配或容量检查、mutex 规则、NUMA 或拓扑约束，也不会改变厂商特定的 `Fit` 行为。这些约束仍保留现有的优先级；当拓扑候选项在其他方面相同时，可以使用利用率得分顺序作为决胜条件。 + +#### Binpack + +Binpack 优先选择设备利用率得分较高的卡。以下默认权重示例假设每张卡有 10 个虚拟设备槽位，并且当前恰好有 1 个槽位正在使用： ```text -GPU1 Score: ((20+10)/100 + (1000+2000)/8000)) * 10 = 6.75 +score: ((request.slot + used.slot) / allocatable.slot + + (request.core + used.core) / allocatable.core + + (request.mem + used.mem) / allocatable.mem) * 10 ``` +1. GPU1 的 Binpack 评分信息如下 + + ```text + GPU1 Score: ((1+1)/10 + (20+10)/100 + (1000+2000)/8000) * 10 = 8.75 + ``` + 1. GPU2 的 Binpack 评分信息如下 -```text -GPU2 Score: ((20+70)/100 + (1000+6000)/8000)) * 10 = 17.75 -``` + ```text + GPU2 Score: ((1+1)/10 + (20+70)/100 + (1000+6000)/8000) * 10 = 19.75 + ``` 因此，在 `Binpack` 策略中我们可以选择 `GPU2`。 #### Spread -Spread 主要关注每张卡的计算能力和显存使用情况。使用越少，得分越高。 +Spread 优先选择设备利用率得分较低的卡。使用相同的默认权重示例： ```text -score: ((request.core + used.core) / allocatable.core + (request.mem + used.mem) / allocatable.mem)) * 10 +score: ((request.slot + used.slot) / allocatable.slot + + (request.core + used.core) / allocatable.core + + (request.mem + used.mem) / allocatable.mem) * 10 ``` 1. GPU1 的 Spread 评分信息如下 -```text -GPU1 Score: ((20+10)/100 + (1000+2000)/8000)) * 10 = 6.75 -``` + ```text + GPU1 Score: ((1+1)/10 + (20+10)/100 + (1000+2000)/8000) * 10 = 8.75 + ``` 1. GPU2 的 Spread 评分信息如下 -```text -GPU2 Score: ((20+70)/100 + (1000+6000)/8000)) * 10 = 17.75 -``` + ```text + GPU2 Score: ((1+1)/10 + (20+70)/100 + (1000+6000)/8000) * 10 = 19.75 + ``` 因此，在 `Spread` 策略中我们可以选择 `GPU1`。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/userguide/configure.md b/i18n/zh/docusaurus-plugin-content-docs/current/userguide/configure.md index 3c8816922..8df4a874a 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/userguide/configure.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/userguide/configure.md @@ -83,6 +83,7 @@ helm install hami hami-charts/hami --set devicePlugin.deviceMemoryScaling=5 -n k | `nvidia.com/use-gputype` | 字符串 | 如果设置了此字段，则该 Pod 分配的设备**必须**是此字符串中定义的 GPU 类型之一。 | `"Tesla V100-PCIE-32GB, NVIDIA A10"` | | `hami.io/node-scheduler-policy` | 字符串 | GPU 节点调度策略：`"binpack"` 表示将 Pod 分配到已有负载的 GPU 节点上执行，`"spread"` 表示分配到不同的 GPU 节点上执行。 | `"binpack"` 或 `"spread"` | | `hami.io/gpu-scheduler-policy` | 字符串 | GPU 卡调度策略：`"binpack"` 表示将 Pod 分配到同一块 GPU 卡上执行，`"spread"` 表示分配到不同的 GPU 卡上执行。 | `"binpack"` 或 `"spread"` | +| `hami.io/device-scoring-weights` | 字符串 | 物理设备评分中虚拟设备槽位、设备核心和设备显存利用率的相对权重。必须提供全部三个权重，值必须为非负整数，并且至少有一个权重大于零。 | `"slot=1,core=1,memory=3"` | | `nvidia.com/vgpu-mode` | 字符串 | 指定该 Pod 希望使用的 vGPU 实例类型。 | `"hami-core"` 或 `"mig"` | ## 容器配置：环境变量