Skip to content

doc: update range grammar description in GPU virtualization #792

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Junie06 wants to merge 1 commit into
base: master
Choose a base branch
from
Open

doc: update range grammar description in GPU virtualization #792

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/core-concepts/gpu-virtualization.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ sidebar_label: GPU Virtualization

In AI inference scenarios, a common dilemma is that GPUs are expensive, but mostly idle.

A typical inference service often only uses 20%~40% of the GPU's compute and a small amount of VRAM, leaving the rest idle. Kubernetes' default GPU scheduling model is exclusive: `nvidia.com/gpu: 1` means the entire card is yours, and all other Pods must wait. Want to share a single GPU across multiple inference services? The standard Device Plugin cannot do it, because it can only report device counts (integers) to the scheduler - there is no concept of "VRAM quota."
A typical inference service often only uses between 20% and 40% of the GPU's compute and a small amount of VRAM, leaving the rest idle. Kubernetes' default GPU scheduling model is exclusive: `nvidia.com/gpu: 1` means the entire card is yours, and all other Pods must wait. Want to share a single GPU across multiple inference services? The standard Device Plugin cannot do it, because it can only report device counts (integers) to the scheduler - there is no concept of "VRAM quota."

@mesutoezdil mesutoezdil Aug 20, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the exact same sentence exists in versioned_docs/version-v2.9.0/core-concepts/gpu-virtualization.md line 8, which is the default version readers see. apply the fix there too. the zh copy needs no change since the tilde range is natural in chinese, worth noting that in the checklist.

@Junie06 Junie06 Aug 20, 2026 •
edited
Loading

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the review @mesutoezdil , the contributing docs advises to edit just docs/, and coderabbit has also just flagged changing the versioned docs, so I reverted the changes

@mesutoezdil mesutoezdil Sep 2, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

checked the rest of the repo: this is the only numeric tilde range in the english docs, and the zh mirror keeps 20%~40%, which is the normal chinese form. so nothing else needs the same edit.


This led to various GPU sharing solutions. NVIDIA's official Time-Slicing allows multiple Pods to be scheduled concurrently, but provides no VRAM isolation - a Pod OOM can crash all tasks on the card. MIG hardware partitioning offers true isolation, but only datacenter-grade cards like A100 and H100 support it.

Expand Down