feat(h1-nvidia): build custom vLLM images on the target host

hosts/h1-nvidia/AGENTS.md

@@ -64,12 +64,16 @@ The tracked defaults in `ansible/vars.yml` stay pinned to the 8 GiB
 workstation case so a fresh `make up HOST=h1-nvidia` on a small card still
 works. Do not change them.
 
-The model needs a locally built image: `vllm-inx:26.06-py3-patched` (a
-Blackwell-patched aarch64 build of vLLM 0.22.1 with the
-`Qwen3_5ForConditionalGeneration` arch, `qwen3_xml` tool parser, `qwen3`
-reasoning parser, and `Qwen3_5MTP` module registered). Upstream
-`vllm/vllm-openai:*` does not carry these as of 2026-08. Build it from
-[`ric03uec/system/hosts/inx/vllm/`](https://github.com/ric03uec/system/tree/main/hosts/inx/vllm).
+The model runs on `vllm-inx:26.06-py3-patched` — a thin patch layer
+(`prometheus-fastapi-instrumentator>=8.0.1`) on `nvcr.io/nvidia/vllm:26.06-py3`.
+NGC's base image ships with `Qwen3_5ForConditionalGeneration`, `qwen3_xml`,
+`qwen3`, and `Qwen3_5MTP` already registered; upstream `vllm/vllm-openai:*`
+does not carry these as of 2026-08.
+
+`10-stack.yml` builds the patched image on the target the first time any
+active model references the tag, using `vllm/images/vllm-inx.Dockerfile`.
+You may need `docker login nvcr.io` on the target beforehand if the base
+image is gated for your account.
 
 Vision (image-text-to-text) is out of scope; tracked in
 [ric03uec/system#32](https://github.com/ric03uec/system/issues/32).

hosts/h1-nvidia/ansible/10-stack.yml

@@ -90,6 +90,7 @@
         - {path: litellm, mode: "0755"}
         - {path: models, mode: "0755"}
         - {path: chat-templates, mode: "0755"}
+        - {path: images, mode: "0755"}
       loop_control:
         label: "{{ item.path }}"
 
@@ -116,6 +117,19 @@
       loop_control:
         label: "{{ item | basename }}"
 
+    # Dockerfiles ship to the target so the docker_image build below has a
+    # build context on the host where the daemon lives, not on the control
+    # machine where the playbook does.
+    - name: Copy custom image Dockerfiles to the install root
+      ansible.builtin.copy:
+        src: "{{ item }}"
+        dest: "{{ install_root }}/images/{{ item | basename }}"
+        mode: "0644"
+      with_fileglob:
+        - "{{ engine_root }}/images/*.Dockerfile"
+      loop_control:
+        label: "{{ item | basename }}"
+
     # =========================================================================
     # Secrets — dropped as a template, never overwritten (T5.6)
     # =========================================================================
@@ -163,6 +177,50 @@
       loop_control:
         label: "{{ item.key }}"
 
+    # =========================================================================
+    # Local image builds
+    #
+    # Any active model whose `image:` tag has no registry prefix (no '/' before
+    # the ':') is a locally-built image. We build it here from a matching
+    # Dockerfile under vllm/images/ if the tag is not already present, so a
+    # first-time `make up` on a fresh host produces a runnable stack in one
+    # command. Public images fall through to compose's own pull.
+    #
+    # Naming contract: the file at `vllm/images/<name>.Dockerfile` produces the
+    # tag `<name>:<version>` where the model YAML's `image:` is `<name>:<version>`.
+    # So `vllm-inx:26.06-py3-patched` builds from `images/vllm-inx.Dockerfile`.
+    # =========================================================================
+    - name: Find locally-tagged images referenced by active models
+      ansible.builtin.set_fact:
+        local_images: >-
+          {{ models
+             | map(attribute='image')
+             | select('match', '^[^/]+:[^/]+$')
+             | unique
+             | list }}
+
+    - name: List images already present on the host
+      ansible.builtin.command: docker image ls --format '{% raw %}{{ .Repository }}:{{ .Tag }}{% endraw %}'
+      register: docker_image_list
+      changed_when: false
+      when: local_images | length > 0
+
+    - name: Build any locally-tagged image that is not already present
+      community.docker.docker_image:
+        name: "{{ item.split(':')[0] }}"
+        tag: "{{ item.split(':')[1] }}"
+        source: build
+        build:
+          path: "{{ install_root }}/images"
+          dockerfile: "{{ item.split(':')[0] }}.Dockerfile"
+          pull: true
+      loop: "{{ local_images }}"
+      loop_control:
+        label: "{{ item }}"
+      when:
+        - local_images | length > 0
+        - item not in docker_image_list.stdout_lines
+
     # =========================================================================
     # Render
     # =========================================================================

hosts/h1-nvidia/vllm/images/vllm-inx.Dockerfile

@@ -0,0 +1,20 @@
+# Thin patch layer on top of the NGC vLLM 26.06-py3 image.
+#
+# Why this exists: 26.06-py3 ships fastapi >= 0.137 (which restructured routers
+# into _IncludedRouter wrappers without a .path attribute) together with
+# prometheus-fastapi-instrumentator 8.0.0, which unconditionally reads .path
+# in its middleware. Every HTTP request 500s before reaching vLLM's router.
+#
+# Fixed in prometheus-fastapi-instrumentator 8.0.1 (2026-06-22). Upstream vLLM
+# also fixed via PRs #45594 (fastapi cap) and #45629 (monkey-patch), neither of
+# which is in the 26.06-py3 build.
+#
+# Refs:
+# - https://github.com/vllm-project/vllm/issues/45596
+# - https://github.com/trallnag/prometheus-fastapi-instrumentator/issues/370
+#
+# The ansible playbook builds this on the target host when any active model
+# references `vllm-inx:26.06-py3-patched` and the tag is not already present.
+# Ported verbatim from ric03uec/system/hosts/inx/vllm/Dockerfile.
+FROM nvcr.io/nvidia/vllm:26.06-py3
+RUN pip install --no-cache-dir "prometheus-fastapi-instrumentator>=8.0.1"

hosts/h1-nvidia/vllm/models/qwen3.8-27b-fp8.yml

@@ -45,13 +45,17 @@ notes: |
   specific additions are `engine: vllm` and `tier: 96g`.
 
   Image: `vllm-inx:26.06-py3-patched` is not on any public registry.
-  Build it locally from the Dockerfile shipped in
-  https://github.com/ric03uec/system/tree/main/hosts/inx/vllm — it
-  is a Blackwell-patched aarch64 build of vLLM 0.22.1 with the
-  Qwen3_5ForConditionalGeneration arch, qwen3_xml tool parser,
-  qwen3 reasoning parser, and Qwen3_5MTP module all registered.
-  Upstream `vllm/vllm-openai:*` does not carry these on any tag as
-  of 2026-08.
+  10-stack.yml builds it on the target host from
+  hosts/h1-nvidia/vllm/images/vllm-inx.Dockerfile the first time it
+  sees this tag referenced by an active model. The Dockerfile is a
+  thin patch layer on `nvcr.io/nvidia/vllm:26.06-py3` (pins
+  prometheus-fastapi-instrumentator>=8.0.1); NGC ships that base
+  with Qwen3_5ForConditionalGeneration, qwen3_xml, qwen3, and
+  Qwen3_5MTP already registered. Upstream `vllm/vllm-openai:*` does
+  not carry these on any tag as of 2026-08.
+
+  You may need `docker login nvcr.io` on the target beforehand if
+  NGC gates the base image behind auth for your account.
 
   Architecture (from HF config.json):
     - Qwen3_5ForConditionalGeneration (dense, w/ vision encoder —

hosts/h3-template/ansible/10-stack.yml

@@ -90,6 +90,7 @@
         - {path: litellm, mode: "0755"}
         - {path: models, mode: "0755"}
         - {path: chat-templates, mode: "0755"}
+        - {path: images, mode: "0755"}
       loop_control:
         label: "{{ item.path }}"
 
@@ -116,6 +117,19 @@
       loop_control:
         label: "{{ item | basename }}"
 
+    # Dockerfiles ship to the target so the docker_image build below has a
+    # build context on the host where the daemon lives, not on the control
+    # machine where the playbook does.
+    - name: Copy custom image Dockerfiles to the install root
+      ansible.builtin.copy:
+        src: "{{ item }}"
+        dest: "{{ install_root }}/images/{{ item | basename }}"
+        mode: "0644"
+      with_fileglob:
+        - "{{ engine_root }}/images/*.Dockerfile"
+      loop_control:
+        label: "{{ item | basename }}"
+
     # =========================================================================
     # Secrets — dropped as a template, never overwritten (T5.6)
     # =========================================================================
@@ -163,6 +177,49 @@
       loop_control:
         label: "{{ item.key }}"
 
+    # =========================================================================
+    # Local image builds
+    #
+    # Any active model whose `image:` tag has no registry prefix (no '/' before
+    # the ':') is a locally-built image. We build it here from a matching
+    # Dockerfile under vllm/images/ if the tag is not already present, so a
+    # first-time `make up` on a fresh host produces a runnable stack in one
+    # command. Public images fall through to compose's own pull.
+    #
+    # Naming contract: the file at `vllm/images/<name>.Dockerfile` produces the
+    # tag `<name>:<version>` where the model YAML's `image:` is `<name>:<version>`.
+    # =========================================================================
+    - name: Find locally-tagged images referenced by active models
+      ansible.builtin.set_fact:
+        local_images: >-
+          {{ models
+             | map(attribute='image')
+             | select('match', '^[^/]+:[^/]+$')
+             | unique
+             | list }}
+
+    - name: List images already present on the host
+      ansible.builtin.command: docker image ls --format '{% raw %}{{ .Repository }}:{{ .Tag }}{% endraw %}'
+      register: docker_image_list
+      changed_when: false
+      when: local_images | length > 0
+
+    - name: Build any locally-tagged image that is not already present
+      community.docker.docker_image:
+        name: "{{ item.split(':')[0] }}"
+        tag: "{{ item.split(':')[1] }}"
+        source: build
+        build:
+          path: "{{ install_root }}/images"
+          dockerfile: "{{ item.split(':')[0] }}.Dockerfile"
+          pull: true
+      loop: "{{ local_images }}"
+      loop_control:
+        label: "{{ item }}"
+      when:
+        - local_images | length > 0
+        - item not in docker_image_list.stdout_lines
+
     # =========================================================================
     # Render
     # =========================================================================

tests/template_test.sh

@@ -46,15 +46,18 @@ for template in "${templates[@]}"; do
   section "$template"
 
   # -- T7.1 -----------------------------------------------------------------
-  # Model YAMLs are excluded: the template ships none on purpose. AGENTS.md is
-  # excluded because both have one and the contents are meant to differ.
+  # Model YAMLs are excluded: the template ships none on purpose. Image build
+  # Dockerfiles under vllm/images/ are excluded for the same reason — they are
+  # per-catalog-entry artifacts, not skeleton pieces every host needs. AGENTS.md
+  # is excluded because both have one and the contents are meant to differ.
   missing=0
   while IFS= read -r ref_file; do
     rel="${ref_file#"$REFERENCE"/}"
     [[ -f "$template/$rel" ]] || { bad "T7.1 missing $rel"; missing=1; }
   done < <(
     find "$REFERENCE" -type f \
       -not -path "$REFERENCE/$ENGINE_DIR/models/*" \
+      -not -path "$REFERENCE/$ENGINE_DIR/images/*" \
       -not -name 'AGENTS.md' | sort
   )
   [[ $missing -eq 0 ]] && ok "T7.1 has every file $REFERENCE has"

website/docs/hosts/h1-nvidia.md

@@ -76,10 +76,12 @@ It sits at tier `96g`, so it is only offered on hosts whose probe reports a
 usable budget of 96 GiB or more. On a Spark, `/lmstack:analyze` picks it up
 automatically. The 8 GiB defaults stay untouched for workstations.
 
-The image is not on any public registry — build `vllm-inx:26.06-py3-patched`
-locally from
-[`ric03uec/system/hosts/inx/vllm/`](https://github.com/ric03uec/system/tree/main/hosts/inx/vllm).
-Vision (image-text-to-text) is out of scope; see
+The image is built on the target host by `10-stack.yml` the first time it
+sees `vllm-inx:26.06-py3-patched` referenced by an active model — it's a
+thin patch layer (`prometheus-fastapi-instrumentator>=8.0.1`) on
+`nvcr.io/nvidia/vllm:26.06-py3`, shipped in `vllm/images/vllm-inx.Dockerfile`.
+You may need `docker login nvcr.io` on the target beforehand if NGC's base
+is gated for your account. Vision (image-text-to-text) is out of scope; see
 [ric03uec/system#32](https://github.com/ric03uec/system/issues/32).
 
 ## Things that will bite you