cua_s1: add a native CUDA text worker

[twu3202]

Purpose

Part of #10: the minimal native text/prefill path for Cua-S1 4B 0.2, as discussed on #12. It builds on #13, whose commits are included here until it merges; the #13 worker is the correctness reference it is checked against.

It serves the text adapter on /v1/systemone like the #13 worker (validation, status codes, prompt token ids, answer format) and runs the forward pass on its own CUDA kernels, without Python or PyTorch:

• src/models/cua_s1/native/: the Rust crate omni-cua-s1-native: request handling, tokenization, the Qwen3.5-4B forward pass (weights, buffers, the layer loop) and the 26-row head.
• src/backends/cuda/qwen3_5/: the operations in CUDA C++: RMSNorm variants, the Gated DeltaNet convolution, gates and chunked prefill, rotary embedding, a FlashAttention-2 style attention kernel on tensor cores, and bfloat16 GEMMs through cuBLASLt. build.sh builds libqwen3_5_cuda.so, which the worker loads at run time, so the workspace builds and tests without a CUDA toolkit.
• recipe/cua_s1/native.md and export_text_merged.py: build, export the merged weights, launch.

Choices worth a look:

• The text adapter is merged into the bfloat16 weights beforehand. Norms and elementwise operations round to bfloat16 where Transformers does; attention and the Gated DeltaNet prefill keep some intermediate results in bfloat16, as FlashAttention and flash-linear-attention do.
• Each prompt runs eagerly with cuBLASLt's first-choice GEMM algorithms. CUDA Graphs and timing cuBLASLt's candidate algorithms at startup come in a follow-up PR with their own measurements.
• The worker runs as its own process behind the frontend, like Add Cua-S1 0.2 multimodal CUDA worker with upstream parity #12 and cua_s1: add a text worker that loads Qwen3.5-4B through Transformers #13. Once feat(frontend): add the in-process engine contract and HTTP service #34 and feat(frontend): own the engine in a worker thread with a bounded queue #35 land, it could be served from the frontend's in-process engine instead.

The comparison scripts behind the results below are not part of the change. They are kept at f9ab3fc.

Test Plan

• CI's checks with Rust 1.98.1: cargo fmt --all --check, cargo clippy --workspace --locked --all-targets -- -D warnings, cargo test --workspace --locked and cargo build --workspace --release --locked.
• Kernel tests on the GPU (CUA_S1_CUDA_LIB=$PWD/target/release/libqwen3_5_cuda.so cargo test --release -p omni-cua-s1-native --test kernels -- --ignored): attention at 1 to 2,048 tokens and the chunked Gated DeltaNet prefill, each against a float64 reference.
• Accuracy: the fixed input set from cua_s1: add a text worker that loads Qwen3.5-4B through Transformers #13 (14 requests, 16 questions) sent to this worker and checked against the float32 results of the cua_s1: add a text worker that loads Qwen3.5-4B through Transformers #13 worker.
• The previous version of this PR, run eagerly, and this one on the same 46 requests (the input set, a case for every error).
• The export script run again, and its output compared file by file with the export used here.
• Latency: bench_text.py over HTTP, one request at a time, 3 warmup and 20 measured requests per case, two runs each.

Environment: one RTX 6000 Ada (48 GB, sm_89), CUDA 13.2, driver 595.91.07, Rust 1.98.1, with the pinned revisions.

System1-Omni Version / Commit: 8367333 on #13 (cfbccc4).

Test Result

• CI's checks pass. Both kernel tests pass: attention is within 4.9e-3 of the float64 reference relative to each row's largest value, and the Gated DeltaNet prefill within 5.8e-4.
• Accuracy: the largest difference from the float32 worker is 0.0057 over the 16 questions, against the allowance of 0.039, and no top option changes.
• Against the previous version run eagerly, every probability is identical (one confidence differs in the last digit, from summing in a different order), and every error gets the same status, except that a body over 4 MiB now gets its 413 where the previous version reset the connection.
• The export script writes files byte-identical to the export used here.
• Startup: 1.9 s to a ready /health; the first request after that took 14 ms. The card peaked at 12.1 GiB in use while serving.
• Latency, p50 in milliseconds. The cua_s1: add a text worker that loads Qwen3.5-4B through Transformers #13 column is from cua_s1: add a text worker that loads Qwen3.5-4B through Transformers #13's description; cua_s1: add a text worker that loads Qwen3.5-4B through Transformers #13 computes logits at every position, which accounts for part of the gap:

Prompt tokens	#13	This PR
139	46.2	18.2
218	49.2	24.8
292	52.7	32.5
712	109.3	68.8
15,446	3084.7	1588.5

Not covered here: score and noul questions and the multimodal adapter; more than one request at a time; GPUs other than sm_89 and CUDA versions other than 13.2.

Self-review

Before marking this PR ready for review or requesting maintainer review, complete
the self-review checklist.
Keep the PR in draft while this work is incomplete.
For agent assistance, use the optional precheck-pr skill.

• I have reviewed the full diff and addressed the issues I found.
• I have checked that the change follows the project's architecture and stays focused on the stated purpose.
• I have run the checks appropriate to this change and reported commands, results, and anything I could not verify above.
• I have checked that the PR description, documentation, and any accuracy or performance claims match the implementation and available evidence.

[xiaoyu-xyz]

Read src/backends/cuda/qwen3_5/ — I'll focus these comments on the CUDA side, since that's the part I'm looking at. Starting with gemm.cu; attention.cu and gdn_prefill.cu next.

What is right, and it is not obvious

• In-place split-K reductions are excluded because their accumulation order, and so the rounding, is not fixed. Giving up some speed for a reproducible result is the right call for an engine that will be checked against a reference, and reduces_in_place re-checks on import rather than trusting the file.
• cs1_gemm_import is all-or-nothing. Every plan is described and checked into checked before any of them replaces a live plan, so a bad file changes nothing.
• The 3 % threshold with "near ties rarely change between runs" is the honest way to handle this: it admits the measurement is noisy instead of chasing it.
• Timing has an L2 flush before each call and takes a median, and a first pass of one call each narrows many candidates to the twelve worth timing properly.
• static_assert(sizeof(cublasLtMatmulAlgo_t) == sizeof(out[n].algo)) pins the ABI assumption at compile time rather than hoping.

Three things I would change

1. plan_for's fallback can adopt an algorithm tuned for a very different M.

if (above && usable(g, p, above->algo)) { p.algo = above->algo; }
else if (below && usable(g, p, below->algo)) { p.algo = below->algo; }

usable() asks whether the algorithm runs for this shape, not whether it is good for it. "The smallest tuned M above, else the largest below" means a shape at M=4096 can end up on an algorithm tuned at M=128 when nothing above has been tuned, with no heuristic fallback and no measurement. The comment describes the selection but not that it is unvalidated — worth either bounding how far the borrowed M may be (say a factor of two) or falling back to the heuristic outside that band.

2. flush_l2 is likely to be optimised away, so the L2 flush before each timed call may not happen.

int acc = 0;
for (...) acc ^= p[i].x ^ p[i].w;
if (acc == 0x7fffffff) *sink = acc;

The store is conditional on a value that essentially never occurs, so the compiler can prove the loop has no observable effect and delete it — at which point g->flush is only ever touched by the cudaMemsetAsync at allocation, and no flush happens inside the timing loop at all. It would not be a correctness problem (this only tunes), but it would make the file's own claim, "times cuBLASLt's candidates for a shape with L2 flushed before every call", untrue, and it would bias the search toward whichever candidate ran first. A store of acc to a sink that is always written, or asm volatile, removes the doubt.

3. Cs1GemmPlan does not carry the cuBLASLt version, and the comment hands that to the caller.

cs1_gemm_version() exists and is exported, but nothing ties the value to the plans. The header says the caller keeps that with the plans, which means the failure mode of forgetting is a silently different (or wrong) algorithm rather than an error. Adding a version field to the struct would be an ABI change, which is what CS1_ABI_VERSION is for — worth doing while the only callers are in this repository.

Smaller notes:

• every_config returns an empty vector when cublasLtMatmulAlgoGetIds fails, which makes cs1_gemm_tune(..., exhaustive=1) silently fall back to the heuristic's shortlist. Returning an error, or logging, would make that visible.
• The log_choice static const bool on = std::getenv(...) is initialised once, so CUA_S1_GEMM_LOG cannot be toggled after the first tuned shape. Fine, just worth knowing.

I have not run any of this — these are from reading, and the two that I would act on first are 1 and 2.

[twu3202]

Thanks, this is useful, especially with Kev planning to load the same library.

1. In this worker every length in TUNE_ROWS is tuned, from 64 to 16384 and never more than 2x apart, and a Cua-S1 prompt is always well over 64 tokens, so the tuned M it borrows from is always within 2x. Only prompts past 16384, if the limit is raised, take the largest one below, and that is deliberate: on sm_89 the heuristic's first pick for long M is a half-rate kernel. Other callers won't have that guarantee, so plan_for now borrows from above only within 2x, from below only when nothing larger was tuned, and otherwise takes the heuristic's choice. Results for this worker are unchanged.
2. The flush does survive compilation: cuobjdump -sass on the built library shows both loads in the loop (LDG.E [R2.64] and [R2.64+0xc]) and the conditional STG.E, since the compiler can't know what the buffer holds. Reading two words of each 16-byte element still pulls every 32-byte sector through L2, so I've left it as is.
3. Agreed. Cs1GemmPlan now carries the cuBLASLt version and import refuses a plan from another version, with CS1_ABI_VERSION at 2. The worker already refused plans files from another cuBLASLt version, but the library shouldn't rely on its caller for that. There's a GPU test for it in tests/kernels.rs.

Also taken: an exhaustive tune now returns an error when cuBLASLt can't list its algorithms. log_choice reading the variable once is intended.

Pushed as 637ced1. With the existing plans file the scores are bitwise unchanged, and a fresh --gemm-search run passes the accuracy check.

[hsliuustc0106]

Reviewed commit 6104d17cc0d879c726002be64f6a8ac3e91e00e6.

No actionable findings in native-worker source review (request mapping, tokenizer export, loading, buffer layout, graph ownership, launch boundaries). Rust library tests: 9 passed, 1 ignored (external float vectors). CUDA kernels and model parity not executed; source review is not numerical certification.

[hsliuustc0106]

cleanup it

[twu3202]

Thanks, cleaned up in a086a31. It drops the comparison and benchmark tooling (diff_corpus.py, diff_workers.py, check_native.py, and the worker's --score-all, --bench and --encode-only modes), the float-vector generator, the Unicode table that made error messages escape names exactly like Python's repr, code only those used, and THIRD_PARTY_NOTICES.md (the notice is now in contract.rs). The diff on top of #13 goes from 8.6k to 6.8k lines, about 0.9k of them Cargo.lock. Statuses and answers are unchanged, and the removed tools are kept at 48ed0e7.

If it helps, once #34 and #35 land I can move serving into the frontend's in-process engine, which would take the HTTP code out of this crate.

[hsliuustc0106]

is this ready for review?

[twu3202]

Yes. I've just pushed the trimmed version: the minimal eager path, with CUDA Graphs and GEMM timing left for a follow-up PR, and the description now has its results.

[twu3202]

For the follow-up: the previous version of this PR captured CUDA Graphs and timed cuBLASLt's candidate GEMM algorithms at startup, without the exhaustive search and plan files. Measured with the same bench on the same GPU, in a separate run, p50 in milliseconds:

Prompt tokens	This PR	With Graphs and GEMM timing
139	18.2	15.7
218	24.8	20.6
292	32.5	27.6
712	68.8	60.0
15,446	1588.5	1324.6

Startup goes from 1.9 s to about 6 s with the timing. I'll rewrite that part on top of this PR, test it again, and open it as a separate PR.

[hsliuustc0106]

generally lgtm, please fix the minors

[hsliuustc0106]

please help review PR #52 as well