Thread swizzling in kernel_mul_mm (Metal) for better cache locality.

[skoulik]

Overview

This adds swizzling trick into the Metal backend's multiplication kernels to avoid hitting the cache size performance cliff on large inputs. Gains 5.5% - 22.5% t/s for Qwen 3.8 27B on Apple M1 Max (depending on ubatch).

Additional information

./llama-bench -m ~./models/qwen3.8-27b/Qwen3.8-27B-Q8_0.gguf -p 8980 -n 0 -ngl 99 -fa on -b 2048 -ub 512,1024,2048 -r 3
ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices
ggml_metal_library_init: using embedded metal library
ggml_metal_library_init: loaded in 0.014 sec
ggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)
ggml_metal_device_init: GPU name: MTL0 (Apple M1 Max)
ggml_metal_device_init: GPU family: MTLGPUFamilyApple7 (1007)
ggml_metal_device_init: GPU family: MTLGPUFamilyCommon3 (3003)
ggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)
ggml_metal_device_init: simdgroup reduction = true
ggml_metal_device_init: simdgroup matrix mul. = true
ggml_metal_device_init: has unified memory = true
ggml_metal_device_init: has bfloat = true
ggml_metal_device_init: has tensor = false
ggml_metal_device_init: use residency sets = true
ggml_metal_device_init: use shared buffers = true
ggml_metal_device_init: recommendedMaxWorkingSetSize = 55662.79 MB

Before:

model	size	params	backend	threads	n_ubatch	fa	test	t/s
qwen35 27B Q8_0	27.11 GiB	27.32 B	MTL,BLAS	8	512	1	pp8980	140.13 ± 0.10
qwen35 27B Q8_0	27.11 GiB	27.32 B	MTL,BLAS	8	1024	1	pp8980	126.28 ± 0.06
qwen35 27B Q8_0	27.11 GiB	27.32 B	MTL,BLAS	8	2048	1	pp8980	117.57 ± 0.11

After:

model	size	params	backend	threads	n_ubatch	fa	test	t/s
qwen35 27B Q8_0	27.11 GiB	27.32 B	MTL,BLAS	8	512	1	pp8980	147.78 ± 0.01
qwen35 27B Q8_0	27.11 GiB	27.32 B	MTL,BLAS	8	1024	1	pp8980	146.01 ± 0.02
qwen35 27B Q8_0	27.11 GiB	27.32 B	MTL,BLAS	8	2048	1	pp8980	143.98 ± 0.07

ctest --output-on-failure -R test-backend-ops
Test project /Users/sergeikulik/CODE/llama.cpp/build/swz
Start 43: test-backend-ops
1/1 Test #43: test-backend-ops ................. Passed 165.43 sec

Perplexity byte-identical on wikitext-2-raw/wiki.test.raw
Final estimate: PPL = 6.7478 +/- 0.10325

Requirements

• I have read and agree with the contributing guidelines
• AI usage disclosure:

YES
Used Claude Opus 5 to debug the inefficiency and suggest the patch.

[ggml-gh-bot]

Hi @skoulik, thanks for your contribution!

Per our contribution guidelines, the automated PR checker found the following issue(s) that need your attention:

• PR Template not respected: Please respect the template when creating a new pull request. Make sure to fill out all required sections.

---

Please note that maintainers reserve the right to make final decisions on PRs. If you believe there is a mistake, please comment below.