vulkan: double the K-quant int-mmq large tile on RDNA3 iGPUs - #27554
Open
aic0d3r wants to merge 1 commit into
Open
vulkan: double the K-quant int-mmq large tile on RDNA3 iGPUs#27554aic0d3r wants to merge 1 commit into
aic0d3r wants to merge 1 commit into
Conversation
The integer-dot mmq path picks its large tile when both matmul dims exceed 64, which is every big GEMM in a dense-model prefill. On RDNA3 iGPUs (gfx1151-class, e.g. Radeon 8060S) the stock 128x128 tile at 256 threads underfills the shader cores at prefill shapes and re-reads the A weights once per 128-token slab; doubling only BN and the thread count keeps the per-thread register tiling identical while halving the A re-reads per token. Qwen3.8-27B UD-Q5_K_XL-v2, RADV, -fa 1 -b/-ub 2048, r=5, same binary stash A/B: pp512 294.5 +-6.4 -> 517.0 +-3.4 (1.76x) pp4096 249.6 +-0.2 -> 436.7 +-0.8 (1.75x) pp16384 216.1 +-0.4 -> 333.2 +-1.7 (1.54x) tg128 10.76 -> 10.82 (unchanged) Decode and verify batches (n <= 64) select the small or medium tile and are unaffected; greedy outputs are byte-identical. wg_denoms are kept in sync with the tile so dispatch and split-k see the real shape. Gated to AMD RDNA3 + integrated GPU via device->uma.
aic0d3r
force-pushed
the
gfx1151-mmq-tile
branch
from
August 22, 2026 17:22
76b0ceb to
cdcf984
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements #27553.
The integer-dot mmq path picks its large tile when both matmul dims exceed 64, which is every big GEMM in a dense-model prefill. On RDNA3 iGPUs (gfx1151-class, e.g. Radeon 8060S) the stock 128x128 tile at 256 threads underfills the shader cores at prefill shapes and re-reads the A weights once per 128-token slab; doubling only BN and the thread count keeps the per-thread register tiling identical while halving the A re-reads per token. No shader changes;
wg_denomskept in sync so dispatch and split-k see the real tile.Qwen3.8-27B UD-Q5_K_XL-v2, RADV,
-fa 1 -b/-ub 2048, same-binary stash A/B, r=5:Also measured (r=3, pp4096): Q4_K_XL 465.1 ±0.8, Q6_K_XL 411.8 ±1.3 with the new tile.
Correctness and scope:
-n 120).AMD RDNA3 && device->uma; discrete GPUs, other vendors, and allmul_mat_id(MoE) tile families untouched.