perf(sm100): stage native MXUMMA output directly - #74
Draft
mgoin wants to merge 1 commit into
Draft
Conversation
Drain TMEM to the existing BF16 shared output layout without the canonical FP32 register transpose. Preserve indexed/TMA writers and bias/secondary-scale rounding; retain generic output for unsupported contracts. Co-authored-by: Codex (GPT-6 Astra) <noreply@openai.com>
This was referenced Sep 6, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on Humming #73. This draft contains only the native MXUMMA output-staging change: 5 files, +127/-3, including 55 lines in the existing kernel tests.
Read accumulators in their native TMEM coordinates, apply the existing BF16 output arithmetic and write the existing swizzled shared-memory output slab. This removes the conversion to the legacy FP32 register layout. Bias and secondary scale rounding are preserved; unsupported arithmetic/output-split configurations use the existing path. Storage sizes, the MMA main loop, routing, output writers and synchronization protocol stay the same.
Validation on B300/SM103 with CUDA 13.0:
The recorded final selection passed 68 native/public/pipeline regressions. Additional checks compared 24 arithmetic cases bitwise against the original cubins, exercised changing-input CUDA graphs, and passed tensor-operation memcheck, racecheck and synccheck with zero reported errors/hazards. Ruff and
git diff --checkpassed. The publication tree is identical to validated combined Humming517cca0; only commit ordering was changed to make this a separate review.Same-input projection measurements use FlashInfer CUPTI, CUDA graphs, cold L2 and three alternating rounds; quantization, packing and communication are excluded. Both implementations pass a dequantized FP32 reference with TF32 disabled (
atol=0.002,rtol=0.008) and match each other bitwise. At fixed BM128/BN128/BK128, four stages and two requested CTAs/SM:Explicit BM64, 16-row Flash guards improved about 1–1.4%; long-K Pro gate was near-neutral/noisy. The no-bias BM128 example removes 320 shuffle sites and lowers registers from 115 to 75, while increasing shared-store work. Actual padded GPT-OSS gate kernels retain their register spills. These measurements do not establish native-backend parity or distributed EP performance.
Real GPT-OSS TP2 and full GSM8K were also checked with this change plus the vLLM activation-fusion companion; combined results belong to that companion description and are not attributed solely to this patch. Current open Humming tuning PRs do not duplicate this epilogue change.
AI assistance: implementation, independent arithmetic/lifetime review and validation used OpenAI Codex, including GPT-6 Astra. Retarget to main after the A8 support dependency merges.