iquant formats: nine weight formats on four tiers, shipped CPU-class tune profiles - #3915
Merged
Conversation
… loader transcode with a decoded 20B scale row, plane pair + every dispatch ladder, portable/reference kernels, repack, tests Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…context codebook fixes, tuner family, image meta; HOW_TO_ADD_A_FORMAT pilot + module CLAUDE.md Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ned dot_lane + the k4 fold), tuner test-mode family, HOW_TO section 5 + QUIRK 11 The kq gemv/tile walks take a third block body: the q40-tiled nibbles decode through the kvalues_iq4nl LUT baked as a constant vector (tbl1/pshufb, the mx4 primitive) into signed int8, dotted on dot_lane's sign-trick lattice, folded as sum_blk sc*idot * d * d8 with no min term. setup_tile_emit wires the mx4 primitive for fmt 44; the decline stubs become the real stamps. gen_tune_probe's test mode gains the 44 family (it was only in the tune-mode list). Gates: tuner family gate 10/10 perms (live stamp dot_maddubs_width256_mr8, maxdiff 1.5e-5), test_kquant -jit 121/125 (4 absent-model skips), test_kqformat 14/14, 1B IQ4_XS end to end text unchanged; after the sidecar re-mint decode 59-60 t/s vs 39 t/s reference bodies. HOW_TO: section 5 rewritten from the executed steps; QUIRK 11 (a sidecar minted while the stubs declined pins "reference" and a .das-only emitter change never invalidates it) and the QUIRK 2 addendum (the probe's two family lists). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ebook decode into signed sdot4 lanes), the id bridge, five-format kernel cells + a float witness; HOW_TO section 6 vk_kq_schema_id gains the 6 -> 44 arm (the arena plan panicked on the first iq4xs stack without it). iq4_word decodes one word of nibble lanes through the kvalues_iq4nl LUT (a fixed_array local the SPIR-V emitter lowers to a Function-storage variable) into signed int8 lanes, so the dots are plain OpSDot with no block-sum term; iq4_sc reads the signed (ls - 32) sub-scale off the k4-shaped 5-word row. The GEMV class overrides blk_contrib, the batch tile stage_w / stage_ws / blk_fma; the five class ladders gain an arm and gemv_cls_set keys on kq_sb. No cm2 decode tile, by followup_vulkan item 24 (new formats land on the class template, not as more hand-stamped bodies) - iq4xs prefill rides the kq batch tile as q40 does. Gates: test_vulkan_kernels 64/64 with the family cells at five formats plus iq4xs_gemv_float_oracle (a dequant straight off the plane bytes - the witness a wrong codebook pack cannot pass while staying class-consistent); the 1B IQ4_XS model on the resident driver reproduces the CPU text at gen 102 t/s. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…er fails closed on IQ4_XS, no q40 precedent, the compact scale-block layout, the ladders a Mac session walks Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…, GEMV + B2/B4/B8 mv twins + the mul_mm IQ4XS arm, the ladders and the gate; tests at fmt 44; HOW_TO section 7 + QUIRKS 12-15 The Metal blob rebakes the iq4xs scale plane into the k6 split form ([nsb x 16B strips = the row's bytes 4..19][nsb x f16 d]), so kq_scales_of, metal_blob_off_ok and every kernel bind are the k6 arms over t.iq4xss; quants bind verbatim. iq4_lut packs the codebook into four words (no table memory, the k6 sign trick widens the byte). Kernels: MetalKqGemvIq4xs, the MetalKqMvIq4xsT B2/B4 pair, MetalKqMvB8Iq4xs, and an IQ4XS arm in MetalKqMulMmK45T (MetalKqMulMmIq4xs). Ladders: enc_kq_gemv / enc_kq_mvb / enc_kq_gemm_mm_b, pf_enc_kq_site_mm (the base mul_mm only), pf_devw_panel_kq (declines), pf_enc_kq_gemv, moe_site_ok + the sb predicates, kq_fmt_gpu_supported. dequant_iq4xs_plane_superblock_at is the split-layout dequant twin the tests and the CPU row both use. Gates on the M1 Max: test_metal_gemv_kernels 2/2 and test_metal_gemm_kernels 2/2 with the iq4xs GEMV / mvb2,4,8 / mul_mm cells; a one-byte codebook mutation turns them red; lint clean. The first run on the box minted run.tune.json (iq4xsq8_tile_gen: mr8, verdict=beats - the emitter's NEON tbl1 path); run.das on the CPU/NEON tier reproduces the text at gen 131 t/s, and the Metal tier (set_metal_mode(required) before load_model_ + convert_model_to_metal_blob) decodes the same text at gen 153 t/s - metal decode/prefill resident paths live on the M1 Max. Not done, ledgered for the M5 kernel pass: the tensor / tall / dev-W mul_mm twins and the MoE GEMV / mul_mm trio for the format. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…d scale row decoded at transcode, [qs][hmask] planes, kernels/repack/tests as k6 twins, every ladder arm; HOW_TO Q3_K note + QUIRKS 16-17 Q3_K is k6's scale structure (16 per-16 signed sub-scales + f16 d, the 18 B row) over a 3-bit quant composed from a 2-bit qs lane and an hmask bit, offset -4 folded like k6's -32. Planes: [qs 64][hmask 32] verbatim (K3_QSB 96); the 12 packed 6-bit scales decode at transcode (q3k_scale6, ggml's kmask unpack) into int8 scale - 32 (K3_SSB 18). dot_k3q8, k3_grp_row_dot, dequant_k3_row_grp, repack_k3_grp (the lanes and mask bits stay in place as 4-byte columns x mr), the k3q8 [tune] family with decline stubs, and every dispatch ladder arm - built by twinning each iq4xs arm of the pilot's CPU-slice commit for k3, which also surfaced two plane-byte accounting sums the pilot had skipped (now counting iq4xs and k3). Gates: test_kqformat 14/14; test_kquant 132/144 interpreted, 140/144 under -jit; the tuner probe's tune-mode family. k3 stays OUT of the probe's test-mode list until its emitter arm lands (QUIRK 16: a stub family fails every perm whose layout companion generates - planes at mr 8, the reference body at mr 4). End to end: Llama-3.2-1B-Instruct-Q3_K_L (Q3_K x64 + Q5_K x48 + Q6_K embd) matches llama.cpp's greedy ids for 52 of 64 tokens on the reference bodies. QUIRK 17: the new [tune] family re-tunes every application sidecar on its first start. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…rms (2-bit lane + hmask compose, offset 4), stamps live, k3 in the probe's test list; HOW_TO note + QUIRK 18 k3 rides emit_block_kqv2 as k6 does (k63 shares the 18 B scale row, the per-16 signed fold and the i16 flush cadence; koff = 4): the compose loads qs column (blk/4)*8 + j (lo) / + 4 (hi), shifts by 2 * (blk % 4), ORs the hmask column j / j + 4 bit blk shifted up two. The decline stubs become kq_gemv_gen_impl / kq_tile_gen_impl at fmt 3; the k3 family joins the probe's test-mode list now that layout and tile decline in lockstep. Gates: the probe 10/10 perms on x64 (dot_maddubs_width256_mr8 live, maxdiff 4.8e-7) and on the M1 Max (sdot mr4/mr8 stamps bit-exact), test_kquant -jit 140/144 on both. The 1B Q3_K_L decodes at 60 t/s after the sidecar re-mint (22 on the reference body); its text flips at token 15 - a near tie: on two real k3 tensors 4957 of 10240 output rows are bit-identical to the reference body and the worst relative difference is 1.7e-4 on a 6e-5 row. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…s with the 2-bit + hmask compose, offset 4), the id bridge, the device gather's k3 arms (and iq4xs's missing tail-row arm), six-format cells + a float witness; HOW_TO note + QUIRK 19 vk_kq_schema_id maps 7 -> 3. KqGemvK3 is k6's blk_contrib over k3_dot (6 uint4 per superblock: qs 0..3 shifted by 2*(blk%4), hmask 4..5 bit blk shifted up two) with the offset 4; KqBatchK3 is a child of KqBatchK6 (stage_ws inherited) composing the staged words and folding with 4. Five class ladders gain an arm, the class oracle picks the class, k3_gemv_float_oracle is the witness off the plane bytes. moe_gpu_gather_stack_kq gains k3 arms in the grouped and the tail-row branches - and the tail-row branch now also copies iq4xs planes verbatim instead of re-pairing nibbles and re-decoding scales (latent: every row of the dense 1B is grouped). Gates: test_vulkan_kernels 64/64 with the family cells at six formats; the 1B Q3_K_L on the resident driver matches llama.cpp's greedy ids for 51 of 64 tokens at gen 284 t/s. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…MetalKqGemvK3 + the B2/B4/B8 mv twins + a K3 mul_mm arm (k6's kernels with the 2-bit + hmask compose, offset 4), the ladders and the gate; tests at fmt 3; HOW_TO note The k3 scale row is already the k6 form, so the blob transform, kq_scales_of and the alignment rule are k6's arms over t.k3s; quants bind verbatim (96 B per superblock: 16 qs uints, 8 hmask uints). Kernels: MetalKqGemvK3 (k6's lane map - a lane's 4 elements of each of a half's 4 blocks share the qs bytes at shifts 0/2/4/6 and hmask bits 4ip..4ip+3), MetalKqMvK3T B2/B4 and MetalKqMvB8K3 (k6's slot map), a K3 arm in MetalKqMulMmK45T beside SIXBIT (MetalKqMulMmK3). Ladders: enc_kq_gemv / enc_kq_mvb / enc_kq_gemm_mm_b, pf_enc_kq_site_mm (base mul_mm), pf_devw_panel_kq (declines), pf_enc_kq_gemv, moe_site_ok, the sb predicates now kq_sb, kq_fmt_gpu_supported. Fixtures at fmt 3 reuse the k6 split-form fill; kq_row_ref dequantizes through dequant_k3_plane_superblock_at. Gates on the M1 Max: test_metal_gemv_kernels 2/2 and test_metal_gemm_kernels 2/2 with the k3 GEMV / mvb2,4,8 / mul_mm cells; lint clean; the 1B Q3_K_L on the Metal tier decodes a coherent story at gen 223 t/s (the other side of the token-15 near tie). Also ledgers followup_general.md `#59`, the partial mint. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…uint4 words instead of a fixed_array local (Function storage); HOW_TO QUIRK 20 + the llama.cpp bench tables for both formats iq4_word held kvalues_iq4nl as a 16-entry fixed_array<int> local and indexed it per nibble; the SPIR-V emitter lowers a dynamically indexed local array to Function storage, and the IQ4_XS decode ran at 105.7 t/s tg128 against k3's 372.8. The codebook now sits in four uint4 words (four int8 codes per word) selected by a dynamic vector index plus a byte shift: 338.4 +- 2.7 t/s against llama.cpp b10660's 340.7 (0.99x), pp512 3363 -> 5161; test_vulkan_kernels 64/64 with the float witness. The Metal twin (iq4_lut) rewritten the same way measured flat on the M1 Max (tg128 144.3 vs 138.7, pp512 2802 vs 2953), so it stays on MSL's constant-address array; the Metal IQ4_XS decode gap is the lane map (followup_general `#58`). HOW_TO: QUIRK 20 (the Function-storage trap, SPIR-V only), the IQ4_XS and Q3_K notes carry the per-tier bench tables against llama.cpp (CPU ahead everywhere, Q3_K GPU decode at or above, Vulkan prefill 0.30x for both = no cm2 decode tile); followup_vulkan item 24 carries the measured prefill numbers as the template's second motivation. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…emory as floats, 16 lanes per superblock, float4 x loads (llama.cpp's kernel_mul_mv_iq4_xs shape) The per-weight constant-array lookup (iq4_lut: two selects, a shift, a mask, the sign fold, the float convert) was the whole gap: MetalKqGemvIq4xs now stages kvalues_iq4nl as 64 floats of threadgroup memory once per group, a 16-lane half of the simdgroup owns one superblock, lane (bu, il) the 8-byte half il of block bu - two uint loads, four float4 x loads, sixteen LUT reads. The gate cell sets the kernel's threadgroup length like the mvb8 cell does. M1 Max, 1B IQ4_XS, lcpp_bench --ngl 99 (das = the debug-jit instrument): tg128 249.4 +- 11.4 against llama.cpp b10660's 249.3 (was 138.7, 0.56x); pp512 unchanged at 2953 (0.83x - the mul_mm arm's turn next). test_metal_gemv_kernels 2/2. Research first (Boris's rule): scratchpad research_llamacpp_iq_kernels.md - llama.cpp keeps every codebook in shared/threadgroup memory pre-converted to float on both backends (PR #10409: IQ4_NL Llama-8B 17.6 -> 59.5 t/s from staging 16 bytes). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…e codebook off threadgroup memory The IQ4XS arm of MetalKqMulMmK45T decoded through the same per-element constant-array lookup the GEMV just shed; the template now carries a @template_gate = IQ4XS threadgroup float[16], staged by the first 16 threads ahead of the tile loop, and the arm indexes it. M1 Max, 1B IQ4_XS, lcpp_bench --ngl 99: pp512 3611.6 +- 5.1 against llama.cpp b10660's 3574.7 (1.01x, was 2953 = 0.83x); tg128 holds 249.3. test_metal_gemm_kernels 2/2. With the GEMV commit this closes the Metal IQ4_XS gap on both axes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…vulkan item 24) KqCm2BatchT carries the whole fast/edge tile body once; the width axis is BN (256/128/32) + STILE (the s column's partial-column fast path, flb rounded to 32, flo Pad) + the BT/ACC/ACCW/ FLO typedefs, the format axis is a template chain - Q8Cm2T/K4Cm2T/K6Cm2T bind BLK/ST/BLKW and author the [spirv_decode] decode - and the nine dispatch classes are eight-line stamps. The bodies are verbatim: the same layouts, the literal no-split loop bounds, the same kernel surface names, so the dispatchers and tests bind unchanged. ~1240 hand-stamped lines fold into ~330; REVIEW_GPU's twin rule stops being violated by this family. Gates on the 5060 Ti: test_vulkan_kernels 63 passed / 1 skipped (the mode-ladder cell, always skipped under DASLLAMA_COOPMAT=cm2) with all six cm2 oracle cells 0-off; the 1B Q4_K_M e2e control row within noise of the hand-stamped baseline - pp512 12997.7 +- 1501.5 against 13144.3 +- 601.2, tg128 358.2 against 360.5. This is the landing pad for the parity work: the iq4xs/k3 (and k5/q40) prefill gap is a missing cm2 tile per format (0.30x llama.cpp on the 1B), and on the template each new format is a decode method, not three kernel copies. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…YLE037 nolint on the mul_mm shell (the IQ4XS staging branch tipped complexity to 21) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… on the tile template Each format is a template stamp now, not three kernel copies: K5Cm2T (the 5th bit off the packed device qh plane), Q40Cm2T (per-block f16 d, the k/k+16 pairing shared with k4's lane view), Iq4xsCm2T (signed sub-scales x super-d, the codebook staged into a gated @Workgroup f16[16] ahead of the tile loop - llama.cpp's init_iq_shmem form, arithmetic only at stage time), K3Cm2T (2-bit lanes + hmask over the k6 scale row). Twelve stamps, the (fmt, tc) dispatch ladders grow their arms, pf_f16_feed admits q8 plus every kq_sb format, and ARCHITECTURE_GPU_VULKAN 2.2k/2.2l now state the seven-format feed. Four device-form CPU oracles and four l/m/s test cells gate it. 5060 Ti, 1B, das = the debug-jit instrument, llama.cpp b10660 Vulkan as the bar: - IQ4_XS pp512 5161 -> 15334 +- 261 (0.90x llama.cpp's 17060, ABOVE the k4 control's 13144); tg128 334.7 holds the GEMV parity (llama.cpp 340.7) - Q3_K_L pp512 5174 -> 14031 +- 86 (0.80x llama.cpp's 17509); tg128 374.5 (llama.cpp 349.1) - suite 72 tests, 71 pass, 1 expected skip (the mode-ladder cell under DASLLAMA_COOPMAT=cm2) The prefill gap for the new formats was the missing cm2 tile (followup_vulkan item 24); what remains against llama.cpp on this 1B shape is the tier-wide small-model gap the k4 control shares. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…s llama.cpp The third format through HOW_TO_ADD_A_FORMAT, and the first grid format: enum iq3s (8), kernel id 33, quant plane [qs 64][qh 8][signs 32] verbatim (104 B), scale row the iq4xs/k4 20 B shape with (1 + 2s) decoded at transcode. The 2 KB iq3s_grid ships as a worker-safe per-call local plus the main-context IQ3S_GRID (the IQ4NL_LUT pattern at 512 entries); signs come off the block's own bytes, so no ksigns table exists anywhere. dot_iq3s_q8 / rows kernel / grp row dequant / repack (26 uniform 4-byte columns) / the ~40 ladders / declined emitter stubs / probe fixtures in both family lists / test arms across every gate (21 mirrored ladders, a pack-direction fixture cell, grid pins). Gates on the worktree binary: test_kqformat 16/16; test_kquant 163 tests, 150 pass, 13 env-gated skips (the batch-groupn class every format shares). E2E: the 1B IQ3_M (IQ3_S x78 + Q4_K x34 + Q6_K embd) matches llama.cpp b10660's greedy ids 64 of 64 on the reference bodies at gen 23 t/s - the first format to hold the whole window. The new family minted verdict=rejected after the whole-scope re-tune, exactly QUIRK 4/16/17's shape. Also: kq_kernel_gen / kq_batch_kernel_gen read the k3 non-repacked rows with k4's strides (128/20 for a 96/18 format) - latent, because every kq load repacks before those branches serve; fixed to 96/18 in passing. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…12 4.93x llama.cpp The first CPU arm written under the llama.cpp side-by-side rule (LAWS 2026-08-30): their kernel's techniques mapped onto our lattice and raced by the probe. The tile rides the panel route - unpack_iq3s_panel_grp expands grid words + signs into the k5/k6 byte panel per group (one i32 store per grid word, sign nibble through a 16-entry mask table, negate (w ^ m) + (m & 0x01010101); magnitudes odd 1..15, no cross-byte carry) - and the gemv gathers each superblock into an alloca panel through an emitted per-row loop (emit_iq3s_gather; the grid and mask live as private [512 x i32]/[16 x i32] module constants). Both ride emit_block_iq4xs's sign-trick lattice with panel loads in place of the nibble+LUT; the reference tile body flips to the panel dot (kq_grp_row_dot_b fmt 33) since the batch cell now hands it the unpacked panel. Also fixed in passing, both found by this arm: - emit_slice recorded the SAVED body/tails block handles as phi incoming blocks; a block-splitting emitter (the gather loop) leaves the builder in its continuation block, so the back edges were malformed. Now captures LLVMGetInsertBlock at the branch points - identical for every non-splitting emitter. - HOW_TO QUIRK 21: an emitter-arm edit does NOT invalidate the JIT DLL cache (generator bodies are not in the hash) - three fix rounds ran stale stamps byte-identically before the cache-hit line gave it away. rm -rf .jitted_scripts after any emitter change. Gates: probe test mode GEN TUNE TEST OK - every k33 perm 1.9e-6 incl the stamped maddubs mr8 pair. Side by side, zen2 16t vs llama.cpp b10660 clean-cpu: pp512 516.9 +- 8.4 vs 104.9 (4.93x - the panel amortizes one gather across the 4-token tile; their nrc=1 kernel re-gathers per token), tg128 52.4 +- 1.0 vs 57.0 (0.92x; the no-panel gemv spelling is followup_general `#61`). Stamped e2e on the 1B IQ3_M: 63/64 greedy ids (the flip is the final token - the stamped near-tie class), gen 44 t/s from 23 on the reference bodies. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…, tg 0.89x The first grid format on the tier. The 2 KB iq3s grid stages into @Workgroup uint[512] once per kernel (llama.cpp's init_iq_shmem form; the source is iq3s_grid_word, a per-index accessor - a shader function cannot return a fixed array - whose QUIRK-20 storage class never sits on a hot path), and the sign nibble expands arithmetically (mask = ((nib * 0x00204081) & 0x01010101) * 255, negate (g ^ mask) + m1). KqGemvIq3s reads the 26-word rows as scalar uints (104 B is not uint4-aligned) with iq4xs's scale fold; KqBatchIq3s : KqBatchIq4xs gathers the grid at stage time and inherits the scale staging and blk_fma verbatim. vk_kq_schema_id 8 -> 33; the batch/gemv ensure/enc ladders grow arms; kq_cls_ref fills the replay's workgroup grid (the CPU replay never runs the kernel head) and the iq3s float witness recomputes rows straight off the plane bytes. Two traps fixed on the way, both now hardened: - pf_f16_feed admitted every kq_sb format, so iq3s prefill hit the cm2 dispatcher whose fall-through silently served the Q8 TILES over iq3s planes - coherent-speed garbage. The feed excludes iq3s until its cm2 tile lands, and all three cm2 ladders verify on any non-q8 fall-through instead of serving the wrong format's pipeline. - the batch family cell's enc ladder lacked the iq3s arm, dispatching the k6 pipeline over iq3s fixtures - 2200/2200 off with byte-stable values across three fix rounds. Gates on the 5060 Ti: test_vulkan_kernels 71/72 (the standard mode-ladder skip) with the seven-format GEMV/batch family cells and the independent iq3s witness; the resident driver matches llama.cpp's greedy ids 64 of 64 at gen 262 t/s. Rows vs llama.cpp b10660 Vulkan: tg128 288.5 +- 0.7 vs 324.2 (0.89x); pp512 6241 +- 9 on the quant feed vs 17865 (0.35x - the cm2 tile on the IQLUT-axis template is the follow-on lever, the same pre-template gap class iq4xs/k3 had at 0.30x). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…12 2x to the tier class The first grid format on the f16 feed: an IQ3GRID gated axis on KqCm2BatchT stages the 2 KB iq3s grid into @Workgroup uint[512] ahead of the tile loop (iq3s_grid_word feeds it, the IQLUT axis's shape at 512 entries), and Iq3sCm2T's decode gathers per element over the 16-bit-lane block views (VkIq3sBlk: qs/qh/signs as int16 arrays) - index = qs byte | qh ninth bit, sign bit off the block's own sign bytes, iq4xs's d x signed (1 + 2s) fold. Three width stamps, cm2_cls_ensure/set/enc arms, and iq3s re-admitted to pf_f16_feed (the previous commit's exclusion lifted now that the format owns a tile). Gates on the 5060 Ti: the three tiles 0-off against the grid-aware f16-feed oracle (89600 cells each, l/m/s); suite 74 tests, 73 passed, 1 standard skip; the resident e2e holds 63/64 greedy ids vs llama.cpp at gen 277 t/s (the flip is the final token - the f16 feed's fp16 activations move the same near-tie the CPU stamp moved; the quant feed ran 64/64). Rows vs llama.cpp b10660 Vulkan: pp512 6241 -> 12539.6 +- 103 (0.70x their 17865 - AT the tier's shared 1B-shape class, the k4 control is 0.67x); tg128 288.1 holds (0.89x). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…tables
A `let` fixed-array local whose initializer is an all-literal fixed_array<T>(...) now lowers
to a program-scope `constant T name[N] = {...};` in the MSL preamble, with the local's reads
indexing it by name - llama.cpp's exact shape for quant codebooks and grids, and the form the
iq3s 512-word grid needs (the previous rule rejected every initializer, and a 2 KB table has
no zero-init-then-fill spelling worth emitting). Same-named tables with different content in
two helpers uniquify off the splice counter; identical content dedupes to one emission.
Anything else with an initializer stays rejected fail-closed: a `var` (a written table has no
constant address space) and any non-literal element, each with its own error text.
Census kind stmt.let.const_table; the ConstTab fixture exercises a hoist from a helper body
and from the kernel body; _fc_array_init_nonconst pins the non-literal rejection needle.
tests/msl: functions 9/9, census, fail_closed all green; the iq3s Metal kernels (the first
consumer) gate 0-off in test_metal_gemv/gemm_kernels on the M1 Max.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…V, tg 0.95x The last tier for the format. Blob: an "iq3ss" arm mirroring the iq4xs 20->18 split verbatim over t.iq3ss (identical strip/d addressing, so kq_scales_of is the iq4xs arm with the plane swapped), quants at sb0*104, the % 512 off gate joining the split-scale row. Kernels ride the previous commit's MSL constant tables: iq3s_gw() carries the 512-word grid, MetalKqGemvIq3s stages it as a float4[512] MAGNITUDE slab (signs applied by select) at 4 rows per simdgroup (llama.cpp's N_R0_IQ3_S), MetalKqMvIq3sT B2/B4 + MetalKqMvB8Iq3s are the iq4xs shells over the per-32-block lane map with the grid read direct, and MetalKqMulMmK45T grows an IQ3S arm (threadgroup grid under @template_gate; the chain re-nested - `} else static_if` is a parse error). Ladders, prefill gates, and the seven-format test cells follow the iq4xs rows; fixtures at fmt 33 reuse the split-form scale fill with kq_row_ref through dequant_iq3s_plane_superblock_at. The decode GEMV got a full race before shipping: eight forms at n=2048 d=8192 (gather placement x3, gather deleted, signs deleted, llama.cpp's exact geometry, row width, f4 slab) all land in 127-141 GB/s against k4's 204 and k6's 287 in the same harness - the 9-bit-index + per-nibble-sign compose is format-intrinsic on M1, and the f4 slab is the best form and ships. HOW_TO gains the race table and QUIRK 22 (judge Metal kernel levers with the kernel probe, not tg128 e2e - +-8 t/s of e2e noise swallowed the +9% winner twice). Gates on the M1 Max: test_metal_gemv_kernels 2/2, test_metal_gemm_kernels 2/2 (108s corpus), e2e decodes the coherent story at gen 227 t/s. Rows vs llama.cpp b10660 Metal (same-run, IQ3_M): pp512 3237.6 / 3344.3 (0.97x), tg128 199.4 / 209.0 (0.95x, up from 0.82x at the first cut); M1 CPU pp512 886.2 / 433.6 (2.04x), tg128 57.4 / 66.6 (0.86x - the ledgered tg tail class, followup `#61`/`#62`). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…d iq3s panel form The fourth format of the arc, identity 34, planes 96/20. The trick that shapes everything: ggml folds w = 0.25 * d * (2ls+1) * grid_byte * sign, and every grid level is EVEN (4..62) - so the plane grid stores the bytes halved (2..31) and the transcode halves the stored d (f16_half_bits: an exponent decrement, exact to the subnormal edge). The 0.25 vanishes with no factor left in any kernel: the fold is iq3s's d x strip x byte verbatim (strip = 2ls+1), and the carry-free smask sign trick still holds (it needs nonzero magnitudes, not odd ones). ksigns_iq2xs (bit 7 = even parity) lands in kqformat, shared with the coming iq2 tier. The panel form is iq3s's: unpack_iq3xxs_panel_grp emits the same signed-byte panel, so kq_grp_row_dot_b, the tile ladders and the batch cell widen to fmt == 33 || 34 instead of growing arms. Everything else is the QUIRK-1 ladder walk: transcode (gguf + superblock), dequant refs, scalar dot + rows kernel, grp repack (24 columns), gemv/tile stubs with the full tune-perm grid (both generators decline until the emitter arc - the framework's own fallback), ~40 dispatch arms, plane pair + image v21, probe fixtures (98B disk blocks), and fmt 34 in every test gate loop. Two finds along the way: the test tile gate's `packed` list wrongly held 33 - the iq3s tile (reference and stamped alike) reads the byte-expanded panel, so the gate handed it packed planes and failed on x64 at HEAD; fixed by handing it the panel like k5/k6. And HOW_TO QUIRK 23: census the vehicle GGUF first - the natural i1-IQ3_XXS download quantizes attn to IQ2_S and cannot load until the iq2 tier; the vehicle is bartowski's Qwen2.5-1.5B IQ3_XS (iq3_xxs attn_k/q + ffn, all siblings supported). Gates: test_kqformat 18/18 (tables, the ksigns parity property, the halver edges), test_kquant 181 tests 0 failed. End to end on the vehicle (reference bodies): coherent text at gen 19 t/s, 22/64 greedy ids vs llama.cpp's simple_ids oracle - the fork is a 0.12-logit near-tie whose runner-up IS our token (simple_ids_margin), the same class as iq3s's token-52 flip on a far less noisy mix. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…route, gen 2.2x The smallest emitter arm of the arc, because Phase A aligned the panel: the tile costs two widened conditions (emit_one_block and emit_block_iq4xs take te.kq == 33 || 34 - the byte-expanded signed panel's stride, scale row and fold are format-agnostic), and only the gemv gather is new code. emit_iq3xxs_gather is the iq3s gather with the aux32 sign word loaded as ONE aligned i32 per block (the four aux bytes sit contiguous in the grp column layout), sign bytes via a dasllama.iq3xxs.ksigns [128 x i32] global feeding the same smask nibble expansion, and the bare qs byte indexing the [256 x i32] HALVED grid - no 9th bit. iq3xxs_emit_globals also shares dasllama.iq3s.smask. The declined stubs flip to kq_gemv_gen_impl/kq_tile_gen_impl(gc, 34). The probe needed the format in TWO lists - the test-mode fixture families and the tune loop's fixed_array - or the gate stamps the perms and silently gates none of them (the first run's log had zero k34 rows next to "stamping 10 permutations"); HOW_TO's Phase-B note carries the trap. Gates on the zen2: 11/11 k34 perms ok vs the scalar oracle (reference, mr4/8, maddubs, vpdpbusd; maxdiff ~2e-6), GEN TUNE TEST OK; the tuner crowns dot_maddubs_width256_mr8 verdict=beats (the board's usual winner, same as iq3s/k3). E2e on the Qwen2.5-1.5B IQ3_XS vehicle: prefill 6 -> 17 t/s, gen 19 -> 41 t/s vs the reference bodies; 17/64 greedy ids vs llama.cpp with the fork ON the margin oracle's top-2 (0.43 logits at step 17 - the stamped float folds move the near-tie flip point from the reference run's step 22, same benign class). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ksigns table The iq3s walk compressed to its minimum. vk_kq_schema_id 9 -> 34. The 1 KB HALVED grid stages into @Workgroup uint[256]; the ksigns table disappears entirely - bit 7 of the sign byte is the even parity of the 7-bit index, five ALU ops (the SPIR-V emitter has no `^=`, so the fold spells as plain assignments). KqGemvIq3xxs rides KqGemvBase over the 24-word rows (one aligned aux32 per block, iq3s's iq4_sc scale fold - the (2ls+1) strips read identically); KqBatchIq3xxs : KqBatchIq4xs gathers the halved grid at stage time; Iq3xxsCm2T lands on a new IQ3XGRID gated axis with the aux 7-bit field picked from at most two unpack8 bytes per element. Dispatcher arms, family cells, the independent float witness and the l/m/s cm2 cell mirror the iq3s rows. The vehicle needed care twice (HOW_TO carries both): QUIRK 23's census caught IQ2_S attn in the natural download, and the Qwen fallback proves kernels but cannot arm the resident decode driver (attention QKV bias declines silently) - so the e2e/bench vehicle is a local requant of Llama-3.2-1B with --tensor-type attn_k/q=iq3_xxs overriding llama.cpp's own IQ2_S recipe, quantized with bartowski's imatrix. Gates on the 5060 Ti: the suite 76/76 - the three cm2 tiles 0-off (89600 cells each), the family cells and float witness green. The resident driver arms on the llama-arch vehicle and matches llama.cpp's greedy ids 32/64 with the fork a 0.093-logit near-tie whose runner-up IS our token (the window's smallest margin), gen 202 t/s. Rows vs llama.cpp b10660 Vulkan (same local requant): pp512 12225.7 / 17807.7 (0.69x), tg128 372.1 / 389.9 (0.95x). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… grid, format closed The fourth tier in one day, and the smallest Metal slice of the arc. Blob: the "iq3xxss" arm mirrors the iq4xs 20->18 split verbatim over t.iq3xxss; quants bind at sb0*96; the % 512 off gate and kq_fmt_gpu_supported grow the format. Kernels are the iq3s shapes with three deltas: iq3xxs_gw (the 1 KB HALVED grid through the constant-table hoisting), ksign7m (the parity sign byte - the ksigns table exists on no GPU tier), and 24-word rows with one aux word per block. The GEMV ships directly in the crowned iq3s form - float4 magnitude slab, sign selects, 4 rows per simdgroup, dispatch rows/8 (QUIRK 22's race already ran for this shape); MetalKqMvIq3xxsT B2/B4 + MetalKqMvB8Iq3xxs and the IQ3XXS mul_mm arm (a gated uint[256] grid, the static_if chain re-nested once more) follow their iq3s siblings, with every compile/release/enc ladder and test cell mirrored at fmt 34. One cross-tier fix rode along: the CPU dequant/dot strip reads went SIGNED (int8 cast) to match the GPU kernels' decode - real strips are 1..31 where the readings agree, but the shared synthetic test fills carry high-bit bytes and the Metal row-ref would have diverged from the kernels over them. test_kquant/test_kqformat stay green. Gates on the M1 Max: test_metal_gemv_kernels 2/2, test_metal_gemm_kernels 2/2 (108s corpus); the e2e decodes the SAME story stream as the CPU and Vulkan tiers at gen 218 t/s (ttft 106ms). Rows vs llama.cpp b10660 (same local requant): M1 Metal pp512 3224.0 / 3429.9 (0.94x), tg128 213.5 / 227.3 (0.94x); M1 CPU pp512 906.0 / 410.5 (2.21x), tg128 53.5 / 74.0 (0.72x); zen2 CPU pp512 507.2 / 136.0 (3.73x), tg128 56.7 / 72.6 (0.78x - with the M1 CPU row, the arc's ledgered tg-tail class, followups `#60`/`#61`). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…one emitter mode The near-free format, identity 45. IQ4_NL's disk bytes ARE Q4_0's (per 32-block f16 d + 16 nibble bytes, the k/k+16 pairing) - only the nibble semantics differ (kvalues_iq4nl, no offset, no bsum). So the planes are Q40_QSB/Q40_SSB verbatim, the transcode is q40's byte split with the type check swapped, the grp repack ROUTES to repack_q40_grp (not copied), and the tile rides the packed lists - no panel, no gather anywhere. The scalar dot is iq4xs's LUT loop with q40's per-block f16-d fold; dequant refs, rows kernel, grp row dot, the ~40 QUIRK-1 ladder arms, plane pair + image v22, probe fixtures (8 x 18B disk blocks) and fmt 45 in every test gate loop follow the walk. The JIT emitter is one `nl` mode on emit_block_iq4xs: per-block d vectors load with the same load_f16_vec_at helper over q40's scale interleave, the fold moves into the block loop (float per-block fma) in place of the sub-scale integer sum, and the LUT bake widens to te.kq == 44 || 45. QUIRK 16 replayed exactly on the pre-emitter gate (10 perms stamped, maddubs rows mismatching on the declined generators at layout mr 8) - the emitter arm turned all 11 k45 rows green (maxdiff ~2e-6) in the same slice, so this commit ships A and B together. Vehicle: a local --allow-requantize (IQ4_NL needs no imatrix; iq4_nl x94 + q5_K/q6_K siblings, census-checked per QUIRK 23). Gates: test_kqformat 18/18, test_kquant 199 tests 0 failed, the tune probe 11/11 k45 perms ok. E2e stamped: coherent text at gen 60 t/s, 11/64 greedy ids vs simple_ids with the fork a 0.042-logit near-tie (the window's smallest margin by two orders) whose runner-up IS our token. zen2 rows vs llama.cpp clean-cpu: pp512 618.3 / 540.6 (1.14x), tg128 64.2 / 62.2 (1.03x). Vulkan, Metal: pending. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Every piece already existed; this commit only connects them. KqGemvIq4nl = iq4xs's iq4_dot codebook compose (iq4_word, pure ALU) folded by q40's q40_d per-block f16 d - no sub-scale, no bsum term. KqBatchIq4nl : KqBatchIq4xs overrides ONLY stage_ws with q40's per-block d stage; the LUT stage_w and the no-min blk_fma inherit verbatim. Iq4nlCm2T = VkK4Blk (q40's nibble lane view) + the existing IQLUT workgroup axis, with Q40Cm2T's d pick times iq4lut[q]. vk_kq_schema_id 10 -> 45; the family cells grow to nine formats (stride 32 words); the cm2 cell mirrors q40's with the codebook oracle; kq_cls_ref takes a plain class arm (pure-ALU codebook - no workgroup state to fill on replay). Gates on the 5060 Ti: the suite 78/78 - the three cm2 tiles 0-off (89600 cells each), the nine-format gemv/batch families green. The resident driver arms on the local vehicle and runs gen 245 t/s; greedy ids 11/64 vs simple_ids (the near-tie class on this stream - its step-11 margin is 0.042 logits). Rows vs llama.cpp b10660 Vulkan (same vehicle): pp512 15027.4 / 19177.8 (0.78x), tg128 340.7 / 363.6 (0.94x). Metal: pending. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… fold swapped, format closed The last tier, and the smallest Metal slice yet: no blob arm, no off-gate row, no split scale plane - q40's 16B/sb plane of 8 f16 d's IS the device form, bound once at soff (kq_scales_of hands doff = 0), and every kernel reads block bu's d at kdh[blk*8+bu] where iq4xs read per-sb d x strip byte. MetalKqGemvIq4nl, MetalKqMvIq4nlT B2/B4 + MetalKqMvB8Iq4nl are the iq4xs shells over the threadgroup codebook; MetalKqMulMmK45T grows an IQ4NL constant that rides the IQ4XS arm flipping only the dsc read. Ladders, prefill gates, the shapes gate, and the test cells follow at fmt 45 (fixtures: 16B of f16 per-block d's, kq_row_ref through dequant_iq4nl_plane_superblock; the five g_pso_* decls live in dasllama_metal_common.das - the [metal_dispatch] pso= attribute only names them). Gates on the M1 Max: test_metal_gemv_kernels 2/2, test_metal_gemm_kernels 2/2 (116 s corpus); the metal-blob e2e decodes the same stream, gen 285 t/s, greedy ids 11/64 vs simple_ids with the fork = the format's 0.042-logit near-tie at step 11 on every tier. Rows vs llama.cpp b10660 (same requant): Metal pp512 3599.5 / 3774.6 (0.95x), tg128 248.4 / 250.2 (0.99x); M1 CPU pp512 727.1 / 724.1 (1.00x), tg128 134.4 / 133.7 (1.01x). IQ4_NL = done on all four tiers. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…0), the k4/k5 fold at per-16 granularity, every ladder arm; QUIRK 25 Q2_K is the k4/k5 scale STRUCTURE at k6's granularity: 16 per-16-element sc/min nibble pairs folded as (d*sc)*q - (dmin*mn) - unsigned 2-bit lanes, so the min term rides the activation 16-sums (xbsp), exactly dot_k4q8's shape. Disk block 84B: [16 sc/min bytes][64 qs][f16 d] [f16 dmin]; planes: qs verbatim (K2_QSB 64, k3's lanes minus the hmask), the scale row reordered header-first [d][dmin][16 sc/min bytes] (K2_SSB 20). dot_k2q8, k2_grp_row_dot, dequant_k2_row_grp, repack_k2_grp (16 four-byte columns x mr; [16 sc x mr][mr x 4B header]), the k2q8 [tune] family with decline stubs, and every dispatch ladder arm - built by twinning the k3 Phase A walk over the current tree's iq4nl-terminated ladders. THE TRAP (QUIRK 25): the mnemonic kernel id 2 IS q51's stream-region tag - the first load crashed with k2 regions dispatched down the q51 repack arm (index out of range on the empty q51s plane). k2 streams under code 20 (kq_stream_code), translated back to kernel id 2 at the two dispatch boundaries (stream_repack_one's arm, repack_regions' fk). The stream-code space carries q8=0, mx4=1, q51=2 - a new format's kernel id must dodge all three. Gates: test_kqformat 18/18, test_kquant 216 (200 pass, 16 env-gated skips), lint 0 on the 20-file changed set. E2e: Llama-3.2-1B-Instruct-Q2_K-local.gguf (local requant: Q2_K x64 + Q3_K x32 + Q4_K x16 + Q6_K embd - every sibling supported) decodes a coherent stream at gen 30 t/s reference bodies; greedy ids 19/64 vs simple_ids with the fork a 0.153-logit near-tie (top2 IS our token; the lossiest format of the arc diverges earliest, as expected). Unlike QUIRK 17's precedent no whole-scope re-mint fired on the first e2e - watch at Phase B. JIT emitter, Vulkan, Metal: pending. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…y-parity The iq2s walk with NO extra planes: each u16 qs word carries a 9-bit grid index (TWO staged words out of a 4 KB @Workgroup uint[1024]) and a 7-bit KSIGNS_IQ2XS index whose sign byte ksign7 recomputes - the iq3xxs parity trick, so no sign plane and no qh anywhere on the device path; the per-16 UNSIGNED strips and eighth-ed d ride the iq2s spellings verbatim. KqGemvIq2xs, KqBatchIq2xs : KqBatchK6 (stage_w gathers off wq[wsb * 16u + k] - the u32 word index IS k), Iq2xsCm2T over VkIq2xsBlk (the 32 u16 words as 16-bit lanes) on a new gated IQ2XSGRID axis. vk_kq_schema_id 13 -> 24; twelve-format family cells at stride 16 (uint4-ALIGNED, unlike its grid siblings); a parity-fold float witness plus an iq2xsf16 cm2 oracle. Gates: test_vulkan_kernels 84/84 (the three iq2xs cm2 tiles 0-off at 89600 cells each), lint 0. E2e resident: parity ids 64/64 vs llama.cpp - the full-match stream holds on the THIRD tier. 5060 Ti vs llama.cpp b10660 Vulkan -ngl 99: pp512 14023.3 vs 18320.0 (0.77x, above the 0.70x tier class), tg128 188.7 vs 349.9 (0.54x - the grid-format gemv workgroup re-stage amplified on a 1B; pp healthy isolates it to the gemv; ledgered as followup_vulkan `#35` with the lever list). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The iq2s Metal walk with ksigns-by-parity end to end: iq2xs_gw (1024 words, the same all-literal program-scope-constant hoisting), MetalKqGemvIq2xs + the MetalKqMvIq2xsT B2/B4 pair + MetalKqMvB8Iq2xs + MetalKqMulMmIq2xs (its own IQ2XS static_if arm nested over IQ2S's) - every kernel derives the sign byte via ksign7m off the u16 qs word, so there is no sign plane and no qh anywhere on the device path. The "iq2xss" blob arm is the iq2ss split verbatim (16 strips + 2B d8 tail, 18B/sb) over the SAME 20B row; quants bind as the 64B plane (16 uint words/sb, uint4-aligned). Gates: test_metal_gemv 2/2 + test_metal_gemm 2/2 on the M1 (first try - the iq2s shells carried it), zen2 kquant -jit 246/4 regression, lint 0. E2e --ngl 99: parity ids 64/64 vs llama.cpp - the FIRST format whose full-match greedy stream holds on ALL FOUR tiers. M1 benches: CPU das 746.6/51.6 vs llama.cpp 144.8/101.1 (5.16x/0.51x - the ledgered grid-format CPU tg tail), Metal das 3223.1/180.1 vs 3462.0/208.3 (0.93x/0.86x - the iq2s pp class). Four-tier table: zen2 2.78x/0.70x, vk 0.77x/0.54x, M1 CPU 5.16x/0.51x, Metal 0.93x/0.86x. The --tune-only filter carried both m1 sidecar top-ups (1 of 14 families raced). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
iq3xxs's aux32 machinery over the iq2xs-style u64 grid: enum iq2xxs (14), kernel/stream 25, 66B disk = [f16 d][8 x [4 byte indices into iq2xxs_grid[256]][aux32: four 7-bit ksigns + per-32 ls nibble]]. The 64B qs region rides VERBATIM as the quant plane (aux in-plane, 16 grp columns); the scale row is the iq3xxs shape (d EIGHTH-ed, 8 per-32 (1+2s) strips). Full CPU walk: transcode/dequant/dot/rows/grp dot/ repack, every dispatch ladder, declined emitter stubs, the tune-list row, and test arms across every gate plus a synthetic plane gate recomposing through grid + the shared ksigns. Most of the walk was GENERATED - a twinning script replayed the IQ2_XS Phase A commit's pure-insert hunks under iq2xs->iq2xxs / 24->25 maps (~500 of ~900 lines); hand work was the aux32 codec bodies, the 512-word grid, and the chain tails. The gates caught the three real traps (HOW_TO Phase A note): the PACKED fmt-25 arms in kq_batch_cell_gen and the test tile ladder (the fmt-24 template is a panel arm; the miss segfaults JIT-only), the grid fn's direct fixed_array return (a let-local round trip broke the JIT cmres), and gen_tune_probe registration (the --tune-only re-mint's "no family matches" doubled as the registration gate). Gates: test_kqformat 18/18, test_kquant interp 248 pass / -jit 263 pass, 0 failed, probe GEN TUNE TEST OK, lint 0 (20 files). QUIRK 15 pinned "reference" via --tune-only iq2xxsq8_tile_gen (1 of 15 families, seconds - the filter's first Phase A). E2e: parity ids 64/64 vs llama.cpp - the THIRD consecutive full-match greedy stream. JIT emitter, Vulkan, Metal: pending. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…th one arm kq_grp_row_dot_b grows the fmt-25 strip arm (per-32 UNSIGNED, the iq3xxs shape) and the emitter needs NO new fold at all: (2ls+1) strip bytes stay under 128, so the shared per-32 SExt load reads them exactly - iq2xxs sets `panel` but not the `iq2` split-accumulator flag and rides the iq3s/iq3xxs scale path verbatim. emit_iq2xxs_gather composes each block's aux32 once, then per l a grid-BYTE index doubles into a [512 x i32] private grid; signs ride the shared iq3xxs ksigns global and the shared smask negate. unpack_iq2xxs_panel_grp mirrors it for the tile; the packed lists drop 25; kfxs25 TEST rows return (QUIRK 16 closed for the format). Probe: 11/11 k25 perms ok (maddubs 9.5e-7); the tuner crowns dot_maddubs_width256_mr8 via --tune-only iq2xxsq8_tile_gen (1 of 15 families, per app). Gates: test_kquant -jit 263 pass / interp 248 pass, 0 failed, lint 0. TRAP for the ledger: run.das takes the model as the LAST .gguf argv token - flags after it silently select DEFAULT_MODEL. E2e stamped: the reference 64/64 full match moves to a step-3 fork on a 0.0267-logit top-2 near-tie (das picks the oracle's `#2` - the arc's tightest; stamped folds move flip points, not the class). zen2 16t vs clean-cpu: pp512 519.0 vs 181.6 (2.86x), tg128 61.1 vs 87.3 (0.70x - the grid-format decode tail class, IQ2_XS's numbers exactly). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… grid KqGemvIq2xxs (2 KB workgroup grid, byte indices + ksign7 parity signs), KqBatchIq2xxs : KqBatchIq4xs with the per-32 strip plane and fma fully INHERITED - (2ls+1) strips stay under 128, so the base's signed extract reads them exactly - and Iq2xxsCm2T / VkIq2xxsBlk on a gated IQ2XXSGRID axis; schema 14 -> 25; thirteen-format family cells at stride 16; a parity-fold float witness + an iq2xxsf16 cm2 oracle. SIGN TRAP for the HOW_TO: the family's random-byte tests demand SIGNED per-32 strip reads everywhere - unsigned reads agree on real strips but fork the hash-fill fixtures one tier at a time. Gates: test_vulkan_kernels 86/86, lint 0. E2e resident: the Vulkan stream is 64/64 IDENTICAL to the stamped CPU stream (tier-bit consistency; both carry only the step-3 0.0267 top-2 flip vs llama.cpp). 5060 Ti vs llama.cpp b10660 Vulkan -ngl 99: pp512 14423.2 vs 18572.5 (0.78x, above the 0.70x class), tg128 297.0 vs 421.4 (0.70x - the 2 KB grid re-stage, milder than iq2xs's 4 KB; followup `#35`). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…adder iq2xxs_gw (512 words, program-scope constant), MetalKqGemvIq2xxs + MetalKqMvIq2xxsT B2/B4 + MetalKqMvB8Iq2xxs + MetalKqMulMmIq2xxs (its own IQ2XXS static_if arm) - signs via ksign7m off the block's aux32, ONE per-32 UNSIGNED strip per block (the 8 pad bytes of the 16B strip region ride along unread); the "iq2xxss" blob arm is the iq2xss split verbatim. Gates: metal gemv 2/2 + gemm 2/2 FIRST TRY, zen2 kquant -jit 263/0 regression, lint 0. E2e --ngl 99: the Metal stream is 64/64 IDENTICAL to the stamped CPU stream - ALL FOUR TIERS of IQ2_XXS are bit-consistent with each other, diverging from llama.cpp only at the step-3 0.0267-logit top-2 flip. M1: CPU das 897.4/56.3 vs llama.cpp 139.9/98.4 (6.41x/0.57x), Metal das 3227.6/180.7 vs 3473.3/231.5 (0.93x/0.78x). Four-tier table: zen2 2.86x/0.70x, vk 0.78x/0.70x, M1 CPU 6.41x/0.57x, Metal 0.93x/0.78x. The ELEVENTH kq format lands its fourth tier: the iquant format ladder is COMPLETE. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The 2026-08-31 sidecar audit across zen2/m1/m4/m5: the kq tile races crowned dot_maddubs_width256_mr8 (mr8 on ARM) on every format and every box, and the 20-seat [tuned] utility walk never spread winner-vs-runner-up past 1.4%. The clones only cost mint time, so the grids now carry one seat per ISA tier: - kq tiles 11 -> 4 seats (mr8 / maddubs avx2 / vpdpbusd vnni256 / vnni512); fallback chains end at mr8 (the mr4 crown loses double-digit % at model scale - the probe header's own finding). - q8q8 ~34 -> 23: per-ISA winners + mr8_budget (the declining reference seat, the M5 crown) + the untouched smmla/amx/vpdpbssd legs, every x86 leg now carrying its requires= gate so race-on-unlock can key on it. - [tuned] grid 20 -> 7: plain/u2/vec8/vec8_u2/vec16/vec16_u2 + the vec4_u4 hand row dot_q4 pins (NEON nibble widening). Gates: gen probe TEST mode green over every surviving perm x 16 families; tune mode green, same crowns (maddubs mr8; q8q8 kstep2 confirms as the per-ISA fallback). Pruned probe tile walk: 372 s vs the ~20-minute pre-prune app mints. plans/unquirk_pass.md carries the full end-of-arc plan. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MyYCDutugdc9xL5pd2qcvT
The mac-session ledger the unquirk plan's Phase E fills out. First entry: the five gated smmla seats q8q8 ships have never raced anywhere - macOS returns an empty LLVMGetHostCPUFeatures so i8mm never detects on Apple Silicon - and the order of operations once B2's darwin feature map lands (prove the emitter arms under forced i8mm, race on M5, then the kq-tile smmla arm as the first real mac kernel work; SME after, research first). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MyYCDutugdc9xL5pd2qcvT
…uirk B2/B3) Kernel winners follow instruction sets, not boxes, so a scope can now ship them: [tune_scope(defaults = dir)] names a directory of checked-in <class>.tune-defaults.json profiles (kernels-only - runtime knobs mix device and box-shape values and never travel). An untuned auto/restart start walks tune_class_chain() down from tune_cpu_class(), adopts the first version-matching profile into the app sidecar, and races only the residue: families the profile predates plus ISA-gated families this box unlocks - decided against the features fingerprint every sidecar save now stamps - armed through the existing DAS_TUNE_ONLY machinery. --tune skips adoption outright and stays the full new-silicon walk; the scope resolver seam is consulted after profiles (local answers outrank remote). DAS_TUNE_POLICY=reference is the true kill switch QUIRK 18 asked for: original bodies for [tune] families and [tuned] loop-hint kernels alike - fallback can serve the very stamp under test when the chain's first viable perm is the crown. Also unlocks the dark smmla leg on Apple Silicon: LLVMGetHostCPUFeatures is empty on macOS, so g_target_arm64_i8mm now also consults cpu_supports (sysctl truth) and the target machine appends +i8mm when the host has it - the gated q8q8 smmla seats become raceable on M2+ with no force env. dasLLAMA: the dasllama scope declares defaults=, x86-avx2 profile checked in (minted fresh on zen2 under the pruned grids: full mint 488 s, noise cv <= 0.33%), harness/export_tune_profile.das mints a profile from a sidecar. Gates: new llvm_tune_profiles.das end-to-end (adopt / race-on-unlock with the armed filter / --tune outranks / no-profile baseline) green under -jit; llvm_tune_scope + manifest + requires + modes green; reference policy verified on the scope client (STATUS source=reference); parity cold-start adopts x86-avx2 with ZERO races and reproduces the recorded stamped stream on the IQ2_XXS vehicle; lint 0 across the changed set. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MyYCDutugdc9xL5pd2qcvT
The scope resolves defaults= against the installed dasllama_math_gen.das, so performance/defaults/*.tune-defaults.json must install beside it or an SDK box never adopts and falls back to the tuner it does not ship. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MyYCDutugdc9xL5pd2qcvT
… compile fix (unquirk B4) The two remaining reference profiles, minted fresh under the pruned grids: - arm-neon from the M1 worktree (gen half 734 s, [tuned] half 70 s; mr8 on every tile, the Metal tensor twins lose to simdgroup on M1); - x86-vnni512 from a c7a.4xlarge (EPYC 9R14 zen4): full new-silicon mint 316 s, noise cv <= 0.21% - dot_vpdpbusd_width512_mr16 crowns EVERY tile family and q8q8 takes the bias128/gkstep2 vnni512 stamp. The instance is terminated; the recipe is in the arc memory. tune_kernels.das failed to COMPILE on macOS: the two Metal race halves reach MetalTensorRaceResult through different require paths and their arrays no longer unify into one typed local. A generic absorb helper takes each returned row where it lands - no named element type, same log lines, same crown list. Windows never compiled that static_if arm, so the rot sat unnoticed under the --tune-only mints. followup_metal.md: the smmla note records the actual fix (cpu_supports at the emitter tier) and the post-PR-1 M5 order. plans/unquirk_pass.md: B4 numbers. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MyYCDutugdc9xL5pd2qcvT
…ng on every platform (C1/D1) QUIRK 24: a layout/pack edit left cached images serving stale planes because the image filename hash folded model + schema and nothing about the packing code. PACK_VERSION now lives beside the pack walkers in dasllama_layout.das and folds into every identity through image_identity_of - the ONE spelling the loader, the peek verdicts, the converter's want/have compare and the tests all format through, so schema and pack versions can never be folded two ways. Bump it with the edit; the image re-bakes instead of being mapped. QUIRK 12: stage_jit_backend staged LLVM.dll on Windows only. It now stages lib/LLVM.dll + its .version stamp everywhere and adds bin/lld-link.exe on Windows, so a fresh worktree on macOS/Linux gets -jit without the hand copy. Gates: lint 0 over the six changed files; test_model_image 48 passed / 19 model-gated skips / 0 failed under -jit; dasllama-convert and setup run. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MyYCDutugdc9xL5pd2qcvT
…adopt return gets a shape Per-module harvest of the comments this branch adds (make_pr row 0a0). Landings: - dasLLVM: REVIEW.md gains the TUNE_KNOWN_FEATURES duty and the arm-then-clear DAS_TUNE_ONLY duty; ARCHITECTURE.md sec.4 "Host CPU feature truth on aarch64" (cited from init_jit_target_flags and create_default_target_machine); profile_try_adopt returns bool + a race csv instead of a ""/"*"/csv string; expr_in_features -> requires_ok_in_fingerprint, skipValue -> skipTuneOnlyValue; three docs the diff had left stale corrected; tests/README covers _lib fixtures. - dasLLAMA: REVIEW.md gains the def-not-let team-lane constant rule and the dot_q8q8_laneq4x4-stays-last bench rule; ARCHITECTURE_GPU_VULKAN.md sec.2.2l carries the cm2 fast-path / no-split-arm facts; ARCHITECTURE_GPU.md sec.2.2y "The Metal kq split scale plane" (four citers); followup_general 58 extended to all nine formats' missing mul_mm twins, 59 closed by --tune-only + profiles; smi/sni -> row_half/col_half in the cm2 tile; attribution stripped from 21 comments; bare measured figures name their harness. - utils: the setup.das staging comment compressed; a Windows-only claim D1 falsified deleted. Gates: lint 0 over the 21 touched .das files; probe TEST OK; Vulkan 86/86; dasLLVM profiles + scope tests green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MyYCDutugdc9xL5pd2qcvT
… the row demotion, and the checklists Three real defects the external round surfaced, each fixed red-first: - embed_row and the PLE gather read a Metal-blob model's SPLIT scale plane with the CPU 20B row stride for every split format but k6. One dispatcher now owns the plane-form question - dequant_kq_plane_sb(fmt, kq, ks, sb, metal_blob, ...) in dasllama_convert.das, kq_planes_of(t, fmt) in common - and test_kquant proves the split form reads to the row form's values for the six 20B-row formats (random f16 scale bits can be NaN on both sides, so the compares are NaN-aware). - gguf_read_tensor_f32 panicked on every new superblock disk type, so the non-native transcode fallback could not load them: it now transcodes one superblock into scratch planes and runs the plane dequant - the native load's own two steps - and test_gguf_quant pins reader == native for all nine types. - kq_fmt_row_ok demoted q40 and k5 on non-256 rows but not iq4nl, a 32-block disk format riding the same lattice; test_kqformat pins the demotion. The audits' own landings: profiles carry no engine_sha (export strips it, performance/REVIEW.das sweeps defaults/); LLVM_JIT_CODEGEN_VERSION 0x59 and aarch64 i8mm folded into jit_env_salt (the +i8mm append is an emission input); DAS_TUNE_POLICY=reference announces on every compile and is rejected as a declared value; run_scope_tuner owns the DAS_TUNE_ONLY arm/clear; a --tune-only filter matching nothing exits 3 per half and fails the mint only when both halves say so; CENSUS_NEVER_DISPATCHED names the 45 new Metal classes; the eleven cm2 cells get poison controls and the family guards skip instead of vanishing; test_kquant joins the model-free suite; the msl const-table rename arm gets its census kind, fixture and a tests/metal oracle; the hoist asymmetry is ledgered in dasMetal's ARCHITECTURE.md; the profiles test covers --tune-only convergence, the class chain and the reference policy. dasLLAMA/REVIEW.md splits its placement rules into REVIEW_PLACEMENT.md (LINT027); the dragon round's wording repairs land across eleven checklists and the two tune skills; followup_metal.md carries the per-format vehicle table (which GGUF, where on zen2 and the M1). Gates: lint 0 over the changed set; Markdown ASCII gate clean; REVIEW.das x4 green; test_kquant 265, test_gguf_quant 9, test_kqformat 20, test_model_image 50, Vulkan 86/86, msl 101/101, coverage, probe TEST, dasLLVM suite incl. the seven-case profiles test - all green under -jit. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MyYCDutugdc9xL5pd2qcvT
The tests-route enumeration, the attribution pair's boundary, the team-lane pointer, the record-hygiene census, the suite-listing duplicate, the media-carrier and ASR carve-outs folded into their triggers, the timing-instrument word, the utils duplicate rule, and the over-long wraps. setup.das carries the earlier comment caps. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0136vDWNJ2GZFzxToEQxWj9i
…reserve, the emitter pin, and the lint/format residue The external round's two P2s, each red-first: - the msl emitter renamed a same-named constant table to tbl_c<seq> without checking whether a helper's own table already held that name, so the rename overwrote it and that helper read the wrong constants. The generated name now probes past every held name; the msl fixture gains a helper owning tbl_c0, the test asserts three surviving tables and the tbl_c1 reads, and the tests/metal oracle proves GPU == CPU with the held name in play. - the IQ3_XXS trim arm bare-resized the @exact_size embedding planes where every sibling arm reserves first; past max_unreserved_size that panics before the copy. PERF032 missed it: its reserve check is function-scoped, so an earlier arm's reserve satisfies every later arm of the ladder. The chain's residue: LLVM_JIT_EMITTER_HASH re-pinned for the i8mm emitter change; two files formatted to the gate's spelling; LINT018/LINT020/LINT002/ LINT003 in the new tests; the dragon's round-4 wording repairs (the crown pin named by its setters, the reference-tool pin named by DEFAULT_REF_SHA, the tensor-twin gloss, PSO expanded, the tests-route and locate-rule wording). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0136vDWNJ2GZFzxToEQxWj9i
Contributor
There was a problem hiding this comment.
Pull request overview
This PR expands dasLLAMA’s quantized-weight support (nine additional iquant/K-quant formats across CPU/JIT/Vulkan/Metal tiers) and evolves the tuning framework so an “untuned” start can adopt shipped CPU-class defaults instead of doing a full race (full race remains available via --tune). It also updates the JIT emitter feature detection/caching (notably aarch64 i8mm) and adds/extends tests and review checklists to keep the new behaviors pinned.
Changes:
- Add shipped defaults profiles keyed by CPU feature class, plus
--tune-onlyre-mint filtering andDAS_TUNE_POLICY=referencebehavior/docs/tests. - Extend Metal MSL emission (hoisted literal fixed-array constant tables) and add both emitted-text fixtures and a real-device CPU-oracle test.
- Update
.dlimidentity handling (centralizedimage_identity_of, pack-code version folded into identity) and expand gguf/kq format coverage and validation.
Reviewed changes
Copilot reviewed 94 out of 96 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| utils/REVIEW.md | Routes certain .dlim/gguf identity/build/load changes to dasLLAMA review discipline. |
| utils/mcp/setup.das | Stages JIT backend artifacts (LLVM.dll + version; lld-link on Windows) for new worktrees. |
| utils/mcp/REVIEW.md | Clarifies install-list requirement for shipped SDK-loaded top-level MCP files. |
| utils/dasllama-convert/main.das | Uses centralized image_identity_of for identity reporting and want/have comparisons. |
| tests/msl/test_msl_functions.das | Adds emitted-text assertions for hoisted constant tables and rename behavior. |
| tests/msl/test_msl_fail_closed.das | Adds fail-closed coverage for non-literal fixed-array initializers. |
| tests/msl/REVIEW.md | Adds checklist rule to register new non-fail-closed fixtures in the census list. |
| tests/msl/_msl_common.das | Adds a const-table fixture kernel + census kinds and registers the new census. |
| tests/msl/_fail_closed/_fc_array_init_nonconst.das | New fail-closed fixture for non-literal fixed-array initializer rejection. |
| tests/metal/test_metal_consttab.das | New real-GPU CPU-oracle test for hoisted constant tables behavior. |
| skills/tune.md | Documents shipped defaults profiles, reference policy, and --tune-only semantics. |
| skills/LAWS.md | Records Boris rulings related to shipped defaults profiles and reference policy. |
| skills/internal/llvm_tune_internals.md | Updates internal notes for defaults-profile adoption and env-only reference. |
| plans/jit_compile_time.md | New plan/ledger for JIT compile-time dev loop and invalidation issues. |
| modules/dasMetal/REVIEW.md | Tightens checklist language around emitter capabilities, census kinds, and fail-closed fixtures. |
| modules/dasMetal/metal/msl_emit.das | Implements hoisting literal fixed-array locals to program-scope MSL constant tables (with collision renaming). |
| modules/dasMetal/ARCHITECTURE.md | Notes Metal-only literal fixed-array hoisting as pending parity vs SPIR-V. |
| modules/dasLLVM/tests/test_tuned.das | Adjusts tuned fallback test for the pruned grid (u2). |
| modules/dasLLVM/tests/test_grid.das | Updates expected grid permutation count to the pruned set. |
| modules/dasLLVM/tests/README.md | Expands fixture filename suffix guidance. |
| modules/dasLLVM/tests/llvm_tune_profiles.das | New -jit test rail for defaults-profile adoption, residue racing, --tune-only, and reference. |
| modules/dasLLVM/tests/llvm_tune_profiles_tuner.das | Fake tuner fixture for the defaults-profile test rail. |
| modules/dasLLVM/tests/llvm_tune_profiles_lib.das | Library fixture declaring a defaults-enabled tune scope and a gated seat. |
| modules/dasLLVM/tests/llvm_tune_profiles_client.das | App fixture exercising adopt → residue-race → re-exec behavior. |
| modules/dasLLVM/REVIEW.md | Broadens “host-dependent behavior” checklist rule to include arch/cpu_supports. |
| modules/dasLLVM/ENVIRONMENT.md | Documents reference and DAS_TUNE_ONLY. |
| modules/dasLLVM/daslib/llvm_jit_run.das | Bumps codegen/emitter versions and salts cache keys with i8mm support. |
| modules/dasLLVM/daslib/llvm_jit_common.das | Uses cpu_supports to augment host-feature truth on aarch64; appends +i8mm where supported. |
| modules/dasLLVM/daslib/llvm_env.das | Adds tune_only env-config field and docs. |
| modules/dasLLVM/ARCHITECTURE.md | Documents --tune-only/DAS_TUNE_ONLY and aarch64 host-feature truth union rule. |
| modules/dasLLAMA/tests/test_model_image.das | Switches identity checks to image_identity_of and adds pack-version identity test. |
| modules/dasLLAMA/tests/test_kernel_coverage.das | Marks new iquant-format kernels as never-dispatched by stocked carriers (with justification). |
| modules/dasLLAMA/tests/test_gguf_quant.das | Adds f32 reader coverage for the new superblock disk types vs transcode+dequant oracle. |
| modules/dasLLAMA/tests/run.das | Adds the new test_kquant.das into the suite listing. |
| modules/dasLLAMA/tests/REVIEW.md | Refines test routing/checklist wording (suite inclusion, sentinel filling, controls). |
| modules/dasLLAMA/tests/_metal_kernel_common.das | Extends kq plane synthesis/dequant reference helpers for new formats. |
| modules/dasLLAMA/REVIEW.md | Adds routing to new placement checklist; refines upstream attribution rules; adds new tune feature-name rule. |
| modules/dasLLAMA/REVIEW_PLACEMENT.md | New checklist capturing what-concern-lands-where rules. |
| modules/dasLLAMA/REVIEW_MEASUREMENT.md | Clarifies where served-page numbers are governed; refines “stage figure” vs served-turn measurement wording. |
| modules/dasLLAMA/README.md | Updates supported/unsupported GGUF weight type lists to include new formats. |
| modules/dasLLAMA/performance/REVIEW.md | Refines records/defaults profile rules and introduces defaults-profile pin expectations. |
| modules/dasLLAMA/performance/REVIEW.das | Adds gate checks for shipped defaults profiles under performance/defaults/. |
| modules/dasLLAMA/performance/defaults/x86-vnni512.tune-defaults.json | Adds shipped defaults profile for x86-vnni512 class. |
| modules/dasLLAMA/performance/defaults/x86-avx2.tune-defaults.json | Adds shipped defaults profile for x86-avx2 class. |
| modules/dasLLAMA/performance/defaults/arm-neon.tune-defaults.json | Adds shipped defaults profile for arm-neon class. |
| modules/dasLLAMA/LAWS.md | Records Boris rulings around HOW_TO series and CPU-kernel audit expectations. |
| modules/dasLLAMA/harness/tune_kernels.das | Adds --tune-only filtering, preserves runtime knobs on partial mint, refactors Metal twin logging. |
| modules/dasLLAMA/harness/export_tune_profile.das | New tool to export a minted sidecar into a shipped defaults profile JSON. |
| modules/dasLLAMA/harness/dasllama_tuner.das | Handles “tune-only no match” per-half and reports filter use; fails only if both halves match nothing. |
| modules/dasLLAMA/followup_vulkan.md | Updates Vulkan followup ledger entries with iquant arc completion notes. |
| modules/dasLLAMA/followup_metal.md | New Metal/mac-session followup ledger. |
| modules/dasLLAMA/followup_general.md | Adds/updates followup items for planes, Metal kernels, and tune-only/defaults behavior. |
| modules/dasLLAMA/dasllama/dasllama_vulkan_common.das | Extends kq schema-id mapping for new formats and clarifies comments. |
| modules/dasLLAMA/dasllama/dasllama_tune.das | Prunes tuned grid, adds reference policy behavior. |
| modules/dasLLAMA/dasllama/dasllama_ple.das | Extends PLE embedding table support to new kq formats and centralizes dequant via shared helpers. |
| modules/dasLLAMA/dasllama/dasllama_metal_shapes.das | Extends GPU-supported format predicates and uses kq_sb where applicable. |
| modules/dasLLAMA/dasllama/dasllama_metal_prefill.das | Adds format-specific mm/gemv dispatch arms; declines dev-W dequant for new formats. |
| modules/dasLLAMA/dasllama/dasllama_metal_common.das | Adds PSO slots and plane binding helpers for new formats and split-scale plane offsets. |
| modules/dasLLAMA/dasllama/dasllama_math.das | Adds per-format kq row dispatch function slots and config-driven mr selection for new formats. |
| modules/dasLLAMA/dasllama/dasllama_image.das | Bumps image version, centralizes identity formatting via image_identity_of, expands meta/stream fields. |
| modules/dasLLAMA/dasllama/dasllama_gpu_resident.das | Extends trim logic to new planes and adds per-format trim arms. |
| modules/dasLLAMA/dasllama/dasllama_gemm_schema.das | Makes GEMM_REFERENCE_MR a function; extends kq qsb/ssb mapping and adds packed-plane predicate. |
| modules/dasLLAMA/dasllama/dasllama_config.das | Extends dlim identity with pack-code version and includes new per-format mr fields. |
| modules/dasLLAMA/dasllama/dasllama_blocks.das | Extends bytes-per-weight accounting for new kq formats. |
| modules/dasLLAMA/CMakeLists.txt | Installs shipped defaults profiles into the SDK bundle. |
| modules/dasLLAMA/CLAUDE.md | Adds module-local instructions and required HOW_TO/skill pointers. |
| modules/dasLLAMA/benchmarks/REVIEW.md | Adds timing-rig routing rule and bench-list ordering rule. |
| modules/dasLLAMA/ARCHITECTURE.md | Updates architecture index section span reference for GPU doc. |
| modules/dasLLAMA/ARCHITECTURE_MEASUREMENT.md | Adds content-addressed archive re-stamping rule explanation. |
| modules/dasLLAMA/ARCHITECTURE_GPU.md | Documents Metal kq split-scale plane layout and invariants. |
| modules/dasLLAMA/ARCHITECTURE_GPU_VULKAN.md | Updates Vulkan cm2 decode/feed documentation for broader format coverage. |
| modules/dasLLAMA/ARCHITECTURE_ENGINE.md | Updates kqformat charter to include shared decode-table builders and team-lane-safe shape. |
| CLAUDE.md | Adds routing entry for modules/dasLLAMA/CLAUDE.md. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
… on the diff, the bench-list WHY without the race verb, no routed-from restatement Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0136vDWNJ2GZFzxToEQxWj9i
…tly N elements, integer or float scalar literals Both refusals said 'every element must be a literal', which a bool, half or vector literal contradicts; the fail-closed fixture pins the new text. The accepted set itself is unchanged - no kernel declares such a table, and the hoist fails closed on shapes it does not emit. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0136vDWNJ2GZFzxToEQxWj9i
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 94 out of 96 changed files in this pull request and generated no new comments.
Suppressed comments (1)
Previously missed (1) — in code that hasn't changed since the last review.
modules/dasLLAMA/harness/dasllama_tuner.das:174
TUNE_ONLY_NO_MATCHis duplicated here and intune_kernels.das(both hard-coded to 3). If one side changes, the wrapper/half contract breaks silently. Consider defining this exit code once (e.g., in a small shared harness module required by both halves) and referencing it from both files.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Behavior change: an untuned dasLLAMA start now adopts a shipped CPU-class profile instead of racing; a full race is
--tune.LLVM_JIT_CODEGEN_VERSIONis bumped (aarch64 hosts append+i8mm), so every JIT cache re-emits once.Nine weight formats land on all four tiers - CPU reference, JIT, Vulkan, Metal: IQ4_XS, Q3_K, IQ3_S, IQ3_XXS, IQ4_NL, Q2_K, IQ2_S, IQ2_XS, IQ2_XXS. Each has a bit-exact plane codec against the upstream dequant order, a repack, generated gemv/tile kernels, the Vulkan class and cm2 tile, and the Metal gemv/mv/mm twins. Greedy-id parity against the reference build holds on every format (the last three match 64/64; the earlier ones fork only at documented sub-0.1-logit ties), and the CPU, Vulkan and Metal streams are bit-consistent with each other. Per-format, per-tier throughput is tabulated in HOW_TO_ADD_A_FORMAT.md, which carries the walk itself and a 27-entry quirk ledger; followup_metal.md carries the per-format vehicle table (which GGUF, where on the zen2 box and the M1).
The tune framework changed because minting had become the workflow: the race rediscovered two facts for twenty minutes per app per box. Grids are pruned to one seat per ISA tier, and
[tune_scope(defaults = ...)]ships kernel-winner profiles keyed by CPU feature class (x86-avx2, x86-vnni512, arm-neon are checked in, minted on zen2, a c7a zen4 and the M1). An untuned start adopts its class profile and races only what the profile could not answer - families it predates, and ISA-gated seats this box unlocks that the minting box did not, decided against thefeaturesfingerprint every sidecar save now stamps.DAS_TUNE_POLICY=referenceis the true kill switch (original bodies,[tune]and[tuned]alike, environment-only).--tune-only <family>re-mints one family; the fast dev loop (-module-cache+--jit-split-modules=-1) is documented with numbers.Defects fixed along the way, each with a red-first test: i8mm never detected on Apple Silicon at the emitter tier (macOS returns an empty LLVM host-features string), which left the existing smmla seats dark; the
.dlimidentity carried no pack-code version, so a layout edit could serve stale planes; a Metal-blob model's split scale planes were read with the CPU row stride byembed_rowand the PLE gather for every split format but k6; the non-native f32 fallback panicked on the new disk types; IQ4_NL rows that are not 256-multiples escaped the q8 demotion; the msl emitter's renamed-constant-table branch could overwrite a helper's own table when that helper had named it like a generated one; the IQ3_XXS trim arm bare-resized its@exact_sizeplanes.Where to look:
modules/dasLLVM/daslib/llvm_tune.das(profile adoption,tune_cpu_class, the gated-families string),modules/dasLLAMA/dasllama/dasllama_convert.das(dequant_kq_plane_sb, the one plane-form dispatcher),dasllama_gemm_gen.das(the emitter arms),dasllama_vulkan_classes.das/dasllama_metal_kernels.das(the GPU tiers),plans/unquirk_pass.md(the plan this PR is the first half of).Validation, claims, ledger
Validation
DASLLAMA_GPU=1), the Metal arms with--ngl.run.das --suite kernels7 files / 18 passed (prefill, decode, rope, gemv, misc, attn, gemm);tests/metal/test_metal_consttab.dasdevice half 2/2 (GPU == CPU with a helper owningtbl_c0);llvm_tune_profiles.dasunder-jiton macOS arm 1/1.llvm_tune_profiles.das(seven cases: adopt, race-on-unlock with the armed filter,--tuneoutranks, no-profile baseline,--tune-onlyconvergence in both spellings, the class chain, thereferencepolicy) needs-jitto exercise the generator tier; it short-circuits to pass without it.--tune-onlyno-match exit); at 21d94c4 two P2 (the renamed-table probe, the trim reserve). Every finding fixed red-first; the branch's commit messages were then rewritten message-only (bare ledger#Nto backticked) before the push, so those two shas name the pre-rewrite tips.LLVM_JIT_EMITTER_HASHre-pinned (the+i8mmappend is an emitter change);LLVM_JIT_CODEGEN_VERSION0x59.Claims - stated, not tested
pf_f16_feednow admits every kq superblock format, which also re-routes the pre-existing k5 and q40 onto the cm2 f16-feed prefill path; settled by the resident Vulkan parity runs on the Q4_K_M / Q5_K_M vehicles, not by a checked-in test.dasllama-convert --trimon the new formats is exercised by no gate; the trim arms are format-symmetric copies of the k6 arm, and the IQ3_XXS reserve fix is by inspection against its siblings.[tuned]grid prune (20 seats to 7) rests on the sidecar race tables of zen2, M1, M4 and M5 (winner-vs-runner-up spread never past 1.4%); no in-tree test pins the grid beyond its size.Not done
plans/unquirk_pass.md. The Metal per-format GEMV class templates stay hand-forked on the format axis until that collapse.performance/records/) for the nine formats: deferred to the records re-mint that follows the PERF_LEDGER split; the per-format tables in HOW_TO are bring-up logs with section-level provenance, not cells.plans/jit_compile_time.md), the duplicated numbering in followup_general.md (57-62 twice), the REVIEW.md grooming items the dragon rounds ledgered.reservelicenses every later arm of a ladder (the trim defect slipped through it); a module-global read reachable from a team-lane kernel; a dynamically indexed fixed-array local inside a kernel class; a per-format ladder whose arm set differs from a sibling ladder's;requires=names outsideTUNE_KNOWN_FEATURES. The formatter spells onename : T?;inside a multi-name parameter list asT ?;- the gate's spelling is applied here, the formatter itself is a follow-up.🤖 Generated with Claude Code
https://claude.ai/code/session_0136vDWNJ2GZFzxToEQxWj9i