feat(vllm): support vLLM 0.29 (torch 2.13) + GPU /pooling fixes

[romeokienzler]

Summary

Adds vLLM 0.29 support to gridfm_graphkit.vllm (the /pooling PowerFlow-reconstruction serving path), superseding the 0.26-only extra shipped in 0.9.0.2. Validated end-to-end on an H100.

The current [vllm] extra pins vllm>=0.26,<0.27, which is incompatible with downstreams that have moved to vLLM 0.29 (e.g. algorithm-nexus, whose candidate now uses vllm==0.29.0). This branch makes the serving stack run on 0.29 + torch 2.13.

What changed

Port (94e5b73)

• Bump [vllm] extra to vllm>=0.29,<0.30 and SUPPORTED_VLLM to ">=0.29,<0.30"; version 0.9.0.2 → 0.9.1.
• Widen core torch to >=2.10,<2.14 (vLLM 0.29 hard-pins torch==2.13.0, so the base pin must admit 2.13).
• torch 2.13 has no prebuilt torch-scatter wheel on data.pyg.org yet → add [tool.uv.extra-build-dependencies] torch-scatter = ["torch"] + a torch-2.13.0 find-links entry (source build).
• One hard-breaking API change fixed: ProcessorInputs.get_mm_hashes(model_id) now requires a second hash_algorithm argument (mirrors vLLM's own built-in terratorch.py).

GPU runtime fixes (94094fc) — surfaced only under a real GPU serve on 0.29, not in import/discovery checks:

• Warmup index-out-of-bounds. vLLM 0.29's profile_run and the new flashinfer_autotune both call _dummy_run(max_num_batched_tokens), which derives num_reqs = min(num_tokens, max_num_seqs) and packs several tokens per request, then gathers hidden_states[logit_indices] up to num_tokens-1. forward returned one row per collated graph item (n_items == num_reqs), so the gather ran off the end (IndexKernel.cu:111). Real requests are 1 token + 1 graph, so num_tokens == n_items and this only bit synthetic warmup. Fix: pad the output's leading (token) axis up to positions.shape[0] — identity on real requests.
• IO-processor async dispatch. 0.29's online pooling path calls the synchronous pre_process from inside the running server event loop, but pre_process wrapped the async variant with asyncio.run(), which raises inside a running loop. Fix: move the (purely synchronous) graph-construction logic into pre_process; pre_process_async delegates.

Validation

• 6 vLLM-free CPU tests pass locally (tests/test_vllm_graph_codec.py, tests/test_vllm_pipeline.py).
• GPU /pooling e2e on H100 (vLLM 0.29.0 / torch 2.13.0+cu130): random-weight export → vllm serve --runner pooling --io-processor-plugin gridfm_pf_reconstruction → POST /pooling on a case14 case → HTTP 200, bus predictions [14 × 4], gen predictions [5 × 1].

Serving requires VLLM_USE_V2_MODEL_RUNNER=0 (0.29's default Model Runner V2 does not support the 'plugin' pooling task used by IO-processor plugins; vLLM raises a ValueError naming this env var). This is a deployment flag, not a code change.

Notes for reviewer

@albanpuech — this is the follow-on to #118 (0.26 serving, merged). A 0.29-compatible release to PyPI is needed before algorithm-nexus can repin its candidate off 0.9.0.2.

🤖 Generated with Claude Code

[albanpuech]

Hello @romeokienzler, thanks for the work!
Since we allow python 3.13, it would make sense to run the integration tests again and change the CI to run all workflows with python 3.13, what do you think?

thank you

[romeokienzler]

@romeokienzler thanks for the port — the change is tightly scoped (model API arity, warmup padding, sync IO-processor dispatch, pins) and the commit messages/inline comments explain the why well.

Current state: all 11 checks green (incl. pip-audit, pytests, DCO signed-off), branch is not behind main. It's BLOCKED only on the required maintainer review, so nothing is blocking you here.

What's worth addressing before a maintainer takes it:

• Python 3.13 (re @albanpuech's point): the repo doesn't actually allow 3.13 yet — pyproject.toml still has requires-python = ">=3.10,<3.13", and CI runs on 3.11/3.12 (ci-build.yaml). So moving the integration tests + workflows to 3.13 would first need the requires-python upper bound widened (and ideally a 3.13 classifier). Happy either way, but worth deciding explicitly rather than leaving the pin and the CI matrix out of sync — and note vLLM 0.29 hard-pins torch==2.13.0, so whatever 3.13 image you pick needs a resolvable torch/torch-scatter combo.
• Tests: the two runtime fixes (warmup leading-axis pad; pre_process sync-vs-async) are the kind of regression that's easy to reintroduce. The pad in particular (num_tokens > n_items) is unit-testable on CPU without a GPU serve — a small test asserting forward output rows == positions.shape[0] when items < tokens, plus one that pre_process runs inside a running event loop, would lock these in. CONTRIBUTING §5 asks for coverage of core behavior + edge cases.

No new params/deps that touch examples/config or tests/config, so those YAMLs look unaffected. Disclosure of AI assistance is already in the PR body — appreciated (not required per AI_POLICY.md).

— 🤖 _automated pre-review; a maintainer will follow up_

[romeokienzler]

Metric-shift attribution: it's the PyTorch 2.12 → 2.13 bump, not a code regression

The integration-test metrics drift vs the committed calibration_baseline.json (calibrated on gridfm-graphkit 0.8.0 / torch 2.12.1). To rule out a code regression on this branch, I ran the current code on the old torch and compared three points:

• A = historic 0.8.0 @ torch 2.12.1 (committed intervals; midpoint = historic seed-0 value)
• B = current 0.9.1 @ torch 2.13.0 (this branch's stack)
• C = current 0.9.1 @ torch 2.12.1 (control run, seed 0, deterministic)

Result: C reproduces the historic interval midpoints to 5–8 significant figures on every metric (PBE Mean matches to 8 digits, 0.41403973). Every code-Δ (C−A) ≤ ~3e-6 → the code change is neutral; C sits dead-center inside all 11 historic intervals. The full divergence appears only in B−C (torch varied, code held fixed):

Metric	C: 0.9.1 @2.12	B: 0.9.1 @2.13	torch Δ
PBE Mean	0.414040	0.421700	+1.9%
Avg. reactive res. (MVar)	0.105288	0.099360	−5.6%
RMSE PG (MW)	2.650304	2.677100	+1.0%
Optimality gap (%)	1.578668	1.209870	−23.4%
Qg viol. PV	0.081620	0.119260	+46.1%
Qg viol. REF	0.186572	0.138080	−26.0%
Qg viol. (total)	0.103371	0.123090	+19.1%

Conclusion: the drift is entirely the torch 2.12→2.13 numerics change, not a regression in this branch (which only touches the vllm/ subpackage, not the training path; runs are deterministic). Recalibrating the baseline on the torch-2.13 stack is the correct fix — seed-0 then lands inside the recalibrated CI for all 11 metrics.

[romeokienzler]

@romeokienzler Thanks — the A/B/C control run is exactly the right way to isolate this: holding the code fixed and varying only torch (C vs B) while showing C reproduces the historic seed-0 midpoints to 5–8 sig figs (PBE Mean to 8 digits) cleanly rules out a code regression, consistent with the diff only touching gridfm_graphkit/vllm/ and leaving the training path untouched. That satisfies the AI_POLICY.md "no unverified claims" bar for the metric-shift attribution — nice.

Two small follow-throughs for whoever picks this up:

• Baseline recalibration is a per-machine action, not a commit. calibration_baseline.json is git-ignored by design (CONTRIBUTING → Running the Integration Tests: bounds are machine-specific), so the fix is "recalibrate on the torch-2.13 stack on your own machine," not a file in this PR. The attribution living in the thread is the right artifact — a maintainer just needs to recalibrate locally on the 2.13 stack before trusting the assertions, and the environment-fingerprint warning will already flag the 2.12→2.13 shift for them.
• Branch is now BEHIND main. It wasn't when I first looked; CONTRIBUTING §8 asks for a rebase onto latest main before merge, so a quick git rebase would clear that.

Still-open from before (unchanged): the Python 3.13 CI decision (@albanpuech's point vs the requires-python = ">=3.10,<3.13" pin) and CPU unit tests for the two runtime fixes. All 11 checks remain green. Deferring the merge call to a maintainer.

— 🤖 _automated pre-review; a maintainer will follow up_

[romeokienzler]

@romeokienzler The rebase landed — the branch now sits on top of the latest main (ba57e19) and reads MERGEABLE, so the BEHIND state I flagged is cleared. Diff is unchanged (same 4 files: vllm/model.py, vllm/plugins/pf_reconstruction/io_processor.py, vllm/utils.py, pyproject.toml), and the re-triggered CI is running clean so far (CodeQL/Trivy/Bandit/detect-secrets/DCO green; pytests + pip-audit re-running).

Still open from earlier (no code change since):

• Python 3.13 CI decision (@albanpuech's point vs. the requires-python = ">=3.10,<3.13" pin + 3.11/3.12 CI matrix) — still needs an explicit call before moving workflows to 3.13.
• CPU unit tests for the two runtime fixes — the warmup leading-axis pad (num_tokens > n_items) and the pre_process sync-inside-running-loop path are both testable without a GPU, per CONTRIBUTING §5.

Neither blocks a maintainer from looking now; BLOCKED is just the required review. Deferring the merge call to a maintainer.

— 🤖 _automated pre-review; a maintainer will follow up_