Skip to content

feat(h1-nvidia): add qwen3.8-27b-fp8 catalog entry for DGX Spark - #12

Merged
ric03uec merged 1 commit into
mainfrom
feat/qwen3.8-27b-fp8
Aug 22, 2026
Merged

feat(h1-nvidia): add qwen3.8-27b-fp8 catalog entry for DGX Spark#12
ric03uec merged 1 commit into
mainfrom
feat/qwen3.8-27b-fp8

Conversation

@ric03uec

Copy link
Copy Markdown
Owner

Closes #11.

Ports the working Qwen3.8-27B-FP8 rollout from ric03uec/system#31 (merged as PR #34) into lmstack as a Spark-class catalog entry on h1-nvidia.

What's in this PR

Model YAML. hosts/h1-nvidia/vllm/models/qwen3.8-27b-fp8.yml — copied verbatim from system/hosts/inx/vllm/models/qwen3.8-27b-fp8.yml with two lmstack-required fields added (engine: vllm, tier: 96g). Native 262 144 context, qwen3_xml tool parser, qwen3 reasoning parser, MTP speculative decoding at num_speculative_tokens: 2. 1.85× throughput speedup on the reference host.

Schema — new 96g tier. Added to VALID_TIERS/TIER_CEILING_GIB in tests/validate_models.py and bin/lmstack-classify. Ceiling deliberately at 96 (not 128) — the DGX Spark's usable pool after RESERVE_GIB is ~118 GiB, so a 128 GiB ceiling would never route. 96 fits the 95 GiB model with 1 GiB slack and matches the actual usable budget a Spark reports through the probe.

Schema — T0.10 loosened. The pi extension parity check now validates model IDs against each host's catalog rather than only its currently-active set. Rationale: a pi provider entry is a capability advertisement (what the operator can route to), not a live-serve manifest. Without this change, advertising Qwen3.8-27B in pi-config/extensions/lmstack-h1.ts would fail T0.10 unless we also flipped the tracked active_models — which would break every fresh workstation deploy.

Pi extension. New id: \"Qwen3.8-27B\" entry alongside the existing 7B, with reasoning: true, contextWindow: 262144, maxTokens: 32768.

Docs. AGENTS.md invariant #4 reworded to keep the 8 GB default floor while acknowledging larger catalog entries; hosts/h1-nvidia/AGENTS.md and website/docs/hosts/h1-nvidia.md each gain a "DGX Spark path" section pointing at the local-build requirement for vllm-inx:26.06-py3-patched and the deferred vision issue.

What's not in this PR (and why)

  • Tracked vars.yml unchanged. Bumping vram_budget_gib: 8 → 128 and swapping active_models — as the initial plan suggested — would break every existing workstation user's next make up. Instead, the tracked defaults stay pinned to the 8 GiB workstation case; on a Spark, the skill writes an override under ~/.lmstack/h1-nvidia/vars.yml at probe time, which the validator already honors (see validate_models.py state-dir precedence).
  • Vision / multimodal. Follows ric03uec/system — deferred to system#32, port that when the upstream lands.
  • Render golden for the 4-virtual-model MTP entry. The model is not in the tracked active_models, so tests/render.yml never renders it. Adding a synthetic fixture just to exercise a dormant catalog entry is overreach; skip until the model is actually active on some host's tracked defaults.

Verification

  • make test — 230/0 passed/failed after the changes. New T0.13 (96g) negative test covers the new tier ceiling. Existing T6.2 GB10 unified memory test (nvidia-gb10-unified fixture) now routes to the 96g tier and recommends the 27B; still passes because it only asserts recommended | length > 0.
  • No live-host verification here. The upstream system#31 closure records end-to-end validation on the reference DGX Spark (weights 28.95 GiB, KV cache 1,001,544 tok, 14.3 tok/s with MTP, tool calls + reasoning round-trip). Same image, same flags — the port is a schema translation, not a new deploy.

Test plan

  • Clone on a workstation → make up HOST=h1-nvidia still comes up on the 7B default (defaults untouched).
  • On a DGX Spark → /lmstack:analyze picks up the 96g tier, the skill writes the override, make up serves qwen3.8-27b-fp8 at the four virtual aliases.
  • curl http://h1-nvidia:4000/v1/models lists Qwen3.8-27B, qwen3.8-27b, Qwen3.8-27B-FP8, qwen3.8-27b-fp8.
  • Baseline chat, tool call, and reasoning requests each succeed through LiteLLM.
  • pi routes to Qwen3.8-27B and completes a non-trivial task.

Ports the working Qwen3.8-27B-FP8 rollout from ric03uec/system#31 into
lmstack as a Spark-class catalog entry on h1-nvidia. Fixes #11.

Model file is copied verbatim from system/hosts/inx/vllm/models/
qwen3.8-27b-fp8.yml with the two lmstack-required fields added
(engine: vllm, tier: 96g). Native 262 144 context, tool calling via
qwen3_xml, reasoning via qwen3, MTP speculative decoding (1.85x
throughput on the reference host).

Schema changes:
- Add tier "96g" (ceiling 96 GiB) to VALID_TIERS in the validator and
  TIER_CEILING_GIB in the classifier. Fits a 95 GiB model with 1 GiB
  slack and is picked by hosts with >= 96 GiB usable, so a DGX Spark
  (~118 GiB after RESERVE_GIB) routes to it. The 128 GiB "class" name
  would never match GB10's actual usable pool.
- Loosen T0.10: the pi extension parity check now validates against
  each host's model *catalog*, not only its currently-active set.
  A provider entry is a capability advertisement; rotating
  active_models is a runtime concern. Without this, a Spark-only
  catalog entry could not be advertised in pi-config while the
  tracked workstation defaults keep the 8 GiB active set.

Workstation defaults untouched:
- hosts/h1-nvidia/ansible/vars.yml stays at vram_budget_gib: 8 with
  qwen2.5-coder-7b as the only active model. A fresh clone of a
  workstation still comes up on the 7B. On a DGX Spark the skill
  writes an override under ~/.lmstack/h1-nvidia/vars.yml at probe
  time.

Requires the locally built vllm-inx:26.06-py3-patched image
(Blackwell-patched aarch64 build of vLLM 0.22.1 with
Qwen3_5ForConditionalGeneration, qwen3_xml, qwen3, and Qwen3_5MTP
registered). Upstream vllm/vllm-openai does not carry these as of
2026-08. Vision path deferred (ric03uec/system#32).
@ric03uec
ric03uec merged commit 0415100 into main Aug 22, 2026
2 checks passed
@ric03uec

Copy link
Copy Markdown
Owner Author

Pushed a second commit adding the vLLM Dockerfile and an ansible task that builds the custom image on the target host — nudging the PR ref to pick it up.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add Qwen3.8-27B-FP8 to h1-nvidia (DGX Spark, 128g tier)

1 participant