feat(h1-nvidia): build custom vLLM images on the target host - #13
Merged
Conversation
Ships the Dockerfile that produces vllm-inx:26.06-py3-patched — the image qwen3.8-27b-fp8 needs — inside the lmstack repo, and teaches 10-stack.yml to build it on the target host when any active model references a locally-tagged image and the tag is not already present. Previously the PR pointed users at ric03uec/system/hosts/inx/vllm/ to build the image themselves. That leaks a dependency on a private- looking sibling repo and breaks the "one command brings up the stack" contract. Now `make up HOST=h1-nvidia` on a DGX Spark with qwen3.8-27b-fp8 active builds the image (thin patch on nvcr.io/nvidia/vllm:26.06-py3 that pins prometheus-fastapi-instrumentator>=8.0.1), then starts the stack. Build-context contract: - Dockerfiles live at hosts/<host>/vllm/images/<name>.Dockerfile and produce the tag <name>:<version> where the model YAML's image: is <name>:<version>. Matching-by-tag is a naming rule, not a schema field. - The layout task copies the images/ dir onto the target so docker_image has a build context where the daemon lives. - The build task detects locally-tagged images by "no '/' before the ':'" and skips any tag already present (`docker image ls`). Also: - Sync hosts/h3-template/ansible/10-stack.yml with the h1-nvidia changes so a fresh copy of the template still has the same capabilities. Per-host duplication is deliberate (see AGENTS.md); drift is what T7.2 exists to catch, and the template stays the reference shape. - Exclude images/ from T7.1 alongside models/: image Dockerfiles are per-catalog-entry artifacts, not skeleton pieces a new host must carry. NGC base image note: users may need `docker login nvcr.io` on the target if their account gates nvcr.io/nvidia/vllm:26.06-py3. Documented in AGENTS.md, the model YAML notes, and the website.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #12 (merged). That PR shipped the
qwen3.8-27b-fp8model YAML but told users to buildvllm-inx:26.06-py3-patchedthemselves fromric03uec/system/hosts/inx/vllm/. This PR closes that gap somake up HOST=h1-nvidiaon a DGX Spark produces a runnable stack in one command.What's in this PR
Dockerfile ships with the repo.
hosts/h1-nvidia/vllm/images/vllm-inx.Dockerfile— verbatim port ofsystem/hosts/inx/vllm/Dockerfile. Thin patch onnvcr.io/nvidia/vllm:26.06-py3that pinsprometheus-fastapi-instrumentator>=8.0.1(works aroundfastapi >= 0.137+prometheus-fastapi-instrumentator 8.0.0returning 500 on every request; refs in the Dockerfile).Ansible builds the image on the target. Three new tasks in
10-stack.ymlbetween the layout and render blocks:image:with no/before the:(sovllm-inx:26.06-py3-patchedmatches;vllm/vllm-openai:latestdoesn't).docker image lsto see what's already present.community.docker.docker_imagewithsource: buildto build any missing tag frominstall_root/images/<name>.Dockerfile.Build-context contract. Dockerfiles ship to the target under
install_root/images/alongside models and chat-templates. The build task runs on the target where the Docker daemon lives, using the copied context — the previous "run docker build against a control-host path" would have failed silently.Naming rule. File
vllm/images/<name>.Dockerfileproduces tag<name>:<version>matched to a model YAML'simage: <name>:<version>. No new schema field — the tag itself is the join key.Template stays in sync.
hosts/h3-template/ansible/10-stack.ymlgets the same three tasks. AGENTS.md's "deliberate duplication" invariant means the template tracks the reference host; adding a capability to h1-nvidia without carrying it to h3-template is exactly the drift T7 is meant to catch.T7.1 exclusion.
tests/template_test.shnow excludesvllm/images/*alongsidevllm/models/*— Dockerfiles are per-catalog-entry artifacts, not skeleton pieces every host must ship.What's not in this PR
docker-ceanddocker-compose-plugin; the modern Docker Engine defaults to buildx fordocker build. If anyone hits a case where they needDOCKER_BUILDKIT=0, that's a follow-up.docker login nvcr.ioon the target if their account gates the base image; documented in AGENTS.md, the model YAML notes, and the website. Wiring credentials fromstack.envis scope creep.Test plan
qwen2.5-coder-7bactive →local_imagesis empty, build task skipped,make upunchanged.qwen3.8-27b-fp8active → firstmake upbuildsvllm-inx:26.06-py3-patchedfrom the shipped Dockerfile, secondmake upis a no-op (image already present,docker image lsfilter skips the build).docker image ls | grep vllm-inxon the target reports the built tag after the first run.make test— 230/0. Existing render tests still cover the tracked 7B active set; the new build task has no unit test yet (would require mocking Docker), only integration coverage on a real Spark.