cua_s1: CUDA graphs and GEMM timing for the native worker

[twu3202]

Purpose

The follow-up noted on #19: CUDA Graphs and GEMM algorithm timing for the native Cua-S1 worker. It builds on #19 (and #13), whose commits are included here until they merge; this PR's own change is 7 files, +363/-32. Kept in draft until #19 merges.

• Prompts up to 2,048 tokens run as a CUDA graph captured for their exact length on first use, keeping the 128 most recently used lengths. Longer prompts run eagerly.
• At startup, the worker times cuBLASLt's candidate algorithms (its heuristic shortlist, up to 16) for each GEMM shape at 19 prompt lengths from 64 to 16,384 tokens, and keeps the fastest when it beats the first choice by more than 3%. Other lengths borrow the choice made for a nearby length.

Choices worth a look:

• The timing runs at every start. When candidates are within a few percent of each other, a restart can pick a different one, which moves the probabilities slightly (by up to 0.021 on the second prompt set below, with no top option changing). Saving the choices to a file would make restarts reproducible, for about 150 more lines; I left that out to keep this small.
• There are no new flags: the graph length limit, the number of kept graphs and the timed lengths are constants.

The comparison scripts behind the first results are the ones from #19, kept at f9ab3fc.

Test Plan

• CI's checks with Rust 1.98.1, and the GPU kernel tests from cua_s1: add a native CUDA text worker #19.
• Accuracy: the fixed input set from cua_s1: add a text worker that loads Qwen3.5-4B through Transformers #13 sent to this worker in three separate starts, each checked against the float32 results of the cua_s1: add a text worker that loads Qwen3.5-4B through Transformers #13 worker.
• The three starts compared with each other, and with cua_s1: add a native CUDA text worker #19, on the same 46 requests (the input set, a case for every error).
• Latency: bench_text.py over HTTP, one request at a time, 3 warmup and 20 measured requests per case, alternating with cua_s1: add a native CUDA text worker #19, two runs each.
• A second prompt set for the graphs: 272 screen-like states from 173 to 2,024 tokens, each a different length, plus prompts of exactly 2,047, 2,048, 2,049 and 16,384 tokens. Sent to this PR and to cua_s1: add a native CUDA text worker #19 in two starts each, and to the cua_s1: add a text worker that loads Qwen3.5-4B through Transformers #13 worker in float32 and bfloat16 for the same accuracy rule. In each start: 32 lengths sent once and then three more times; 240 other lengths sent once each, more than the 128 kept graphs; 40 of those again; then 320 of them from 8 clients at once.

Environment: one RTX 6000 Ada (48 GB, sm_89), CUDA 13.2, driver 595.91.07, Rust 1.98.1, with the pinned revisions.

System1-Omni Version / Commit: fc5a59d on #19 (8367333).

Test Result

• CI's checks and the kernel tests pass.
• Accuracy: in each of the three starts, the largest difference from the float32 worker is 0.0126 over the 16 questions, against the allowance of 0.039, and no top option changes.
• Between starts, one or two of the 18 answered requests changed, by at most 0.0083 in a probability, and every status is the same. Against cua_s1: add a native CUDA text worker #19, every status is the same and probabilities differ by at most 0.0103.
• On the second set, both starts pass the same rule: the largest difference from float32 is 0.0187 (0.0172 for cua_s1: add a native CUDA text worker #19) against an allowance of 0.073, and no top option changes. Between the two starts, 148 of the 276 answers changed, by at most 0.021, and no top option changed. cua_s1: add a native CUDA text worker #19 gave byte-identical answers in both starts.
• Startup: 6.2 s to a ready /health (1.9 s for cua_s1: add a native CUDA text worker #19); the first request after that took 16 ms. The card peaked at 12.8 GiB during the timing at startup, and at 13.1 GiB while serving with 128 graphs kept and after a 16,384-token prompt (12.1 GiB for cua_s1: add a native CUDA text worker #19).
• Latency, p50 in milliseconds:

Prompt tokens	#19	This PR
139	18.2	16.1
218	24.6	21.1
292	32.3	28.8
712	68.8	60.1
15,446	1586.0	1477.6

On the second set:

• The first request at a new length, which captures its graph, took 0.4 ms longer on average than the repeats at that length, and at most 5 ms.
• The 240 lengths sent once each averaged 99.5 ms, against 104.7 ms for cua_s1: add a native CUDA text worker #19: 6.5% lower below 512 tokens, 8% lower from 512 to 1,023, and 4% lower from 1,024 to 2,020.
• Lengths whose graphs had been dropped and were captured again, and all 320 requests from 8 clients at once, gave byte-identical answers to the first ones in the same start. The 320 requests took 31.3 s (33.2 s for cua_s1: add a native CUDA text worker #19).
• 2,049 tokens (run eagerly) took 175 ms (188 ms for cua_s1: add a native CUDA text worker #19) and 16,384 tokens took 1,594 ms (1,659 ms for cua_s1: add a native CUDA text worker #19). 16,385 tokens gets a 413, as with cua_s1: add a native CUDA text worker #19.

Not covered here: GPUs other than sm_89 and CUDA versions other than 13.2.

Self-review

Before marking this PR ready for review or requesting maintainer review, complete
the self-review checklist.
Keep the PR in draft while this work is incomplete.
For agent assistance, use the optional precheck-pr skill.

• I have reviewed the full diff and addressed the issues I found.
• I have checked that the change follows the project's architecture and stays focused on the stated purpose.
• I have run the checks appropriate to this change and reported commands, results, and anything I could not verify above.
• I have checked that the PR description, documentation, and any accuracy or performance claims match the implementation and available evidence.

[twu3202]

Closing this one: the graph part landed through #52, and I'd rather not add the startup GEMM timing on its own. It adds about 4 s to startup and makes answers depend on which kernels a given start picked (148 of 276 answers moved by up to 0.021 between restarts in my runs), which doesn't fit the reproducible-by-default path the native worker has now. The branch stays on my fork if it's wanted later.