feat(h1-nvidia): add qwen3.8-27b-fp8 catalog entry for DGX Spark - #12
Merged
Conversation
Ports the working Qwen3.8-27B-FP8 rollout from ric03uec/system#31 into lmstack as a Spark-class catalog entry on h1-nvidia. Fixes #11. Model file is copied verbatim from system/hosts/inx/vllm/models/ qwen3.8-27b-fp8.yml with the two lmstack-required fields added (engine: vllm, tier: 96g). Native 262 144 context, tool calling via qwen3_xml, reasoning via qwen3, MTP speculative decoding (1.85x throughput on the reference host). Schema changes: - Add tier "96g" (ceiling 96 GiB) to VALID_TIERS in the validator and TIER_CEILING_GIB in the classifier. Fits a 95 GiB model with 1 GiB slack and is picked by hosts with >= 96 GiB usable, so a DGX Spark (~118 GiB after RESERVE_GIB) routes to it. The 128 GiB "class" name would never match GB10's actual usable pool. - Loosen T0.10: the pi extension parity check now validates against each host's model *catalog*, not only its currently-active set. A provider entry is a capability advertisement; rotating active_models is a runtime concern. Without this, a Spark-only catalog entry could not be advertised in pi-config while the tracked workstation defaults keep the 8 GiB active set. Workstation defaults untouched: - hosts/h1-nvidia/ansible/vars.yml stays at vram_budget_gib: 8 with qwen2.5-coder-7b as the only active model. A fresh clone of a workstation still comes up on the 7B. On a DGX Spark the skill writes an override under ~/.lmstack/h1-nvidia/vars.yml at probe time. Requires the locally built vllm-inx:26.06-py3-patched image (Blackwell-patched aarch64 build of vLLM 0.22.1 with Qwen3_5ForConditionalGeneration, qwen3_xml, qwen3, and Qwen3_5MTP registered). Upstream vllm/vllm-openai does not carry these as of 2026-08. Vision path deferred (ric03uec/system#32).
Owner
Author
|
Pushed a second commit adding the vLLM Dockerfile and an ansible task that builds the custom image on the target host — nudging the PR ref to pick it up. |
4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #11.
Ports the working Qwen3.8-27B-FP8 rollout from
ric03uec/system#31(merged as PR #34) into lmstack as a Spark-class catalog entry onh1-nvidia.What's in this PR
Model YAML.
hosts/h1-nvidia/vllm/models/qwen3.8-27b-fp8.yml— copied verbatim fromsystem/hosts/inx/vllm/models/qwen3.8-27b-fp8.ymlwith two lmstack-required fields added (engine: vllm,tier: 96g). Native 262 144 context,qwen3_xmltool parser,qwen3reasoning parser, MTP speculative decoding atnum_speculative_tokens: 2. 1.85× throughput speedup on the reference host.Schema — new
96gtier. Added toVALID_TIERS/TIER_CEILING_GIBintests/validate_models.pyandbin/lmstack-classify. Ceiling deliberately at 96 (not 128) — the DGX Spark's usable pool afterRESERVE_GIBis ~118 GiB, so a 128 GiB ceiling would never route. 96 fits the 95 GiB model with 1 GiB slack and matches the actual usable budget a Spark reports through the probe.Schema — T0.10 loosened. The pi extension parity check now validates model IDs against each host's catalog rather than only its currently-active set. Rationale: a pi provider entry is a capability advertisement (what the operator can route to), not a live-serve manifest. Without this change, advertising
Qwen3.8-27Binpi-config/extensions/lmstack-h1.tswould fail T0.10 unless we also flipped the trackedactive_models— which would break every fresh workstation deploy.Pi extension. New
id: \"Qwen3.8-27B\"entry alongside the existing 7B, withreasoning: true,contextWindow: 262144,maxTokens: 32768.Docs.
AGENTS.mdinvariant #4 reworded to keep the 8 GB default floor while acknowledging larger catalog entries;hosts/h1-nvidia/AGENTS.mdandwebsite/docs/hosts/h1-nvidia.mdeach gain a "DGX Spark path" section pointing at the local-build requirement forvllm-inx:26.06-py3-patchedand the deferred vision issue.What's not in this PR (and why)
vars.ymlunchanged. Bumpingvram_budget_gib: 8 → 128and swappingactive_models— as the initial plan suggested — would break every existing workstation user's nextmake up. Instead, the tracked defaults stay pinned to the 8 GiB workstation case; on a Spark, the skill writes an override under~/.lmstack/h1-nvidia/vars.ymlat probe time, which the validator already honors (seevalidate_models.pystate-dir precedence).ric03uec/system— deferred to system#32, port that when the upstream lands.active_models, sotests/render.ymlnever renders it. Adding a synthetic fixture just to exercise a dormant catalog entry is overreach; skip until the model is actually active on some host's tracked defaults.Verification
make test— 230/0 passed/failed after the changes. NewT0.13 (96g)negative test covers the new tier ceiling. ExistingT6.2 GB10 unified memorytest (nvidia-gb10-unifiedfixture) now routes to the96gtier and recommends the 27B; still passes because it only assertsrecommended | length > 0.system#31closure records end-to-end validation on the reference DGX Spark (weights 28.95 GiB, KV cache 1,001,544 tok, 14.3 tok/s with MTP, tool calls + reasoning round-trip). Same image, same flags — the port is a schema translation, not a new deploy.Test plan
make up HOST=h1-nvidiastill comes up on the 7B default (defaults untouched)./lmstack:analyzepicks up the96gtier, the skill writes the override,make upservesqwen3.8-27b-fp8at the four virtual aliases.curl http://h1-nvidia:4000/v1/modelslistsQwen3.8-27B,qwen3.8-27b,Qwen3.8-27B-FP8,qwen3.8-27b-fp8.Qwen3.8-27Band completes a non-trivial task.