ChannelQuant

A streaming hardware codec for transformer KV caches. Status: Phases 0–4 landed — bit-accurate reference model, n=1000 headline + generalization, method paper; the CQ-3-rot WHT value tier is merged (bit-exact here; accuracy screening) · Date: 2026-07-20 · Host: DGX Spark (GB10, Grace-Blackwell); Phase 3–4 runs on H100.

ChannelQuant is a streaming hardware codec for transformer KV caches that implements the KIVI/KVQuant recipe — per-channel INT4 scaling for keys, per-token INT4 for values, with the handful of high-magnitude key channels held in higher precision via a statically-calibrated mask. On Qwen2 GQA models it reaches ~3.6–3.8× compression at near-lossless task accuracy (within 0.01 HellaSwag acc_norm of FP16 up to 7B), where naive INT4 collapses to chance. Values now drop further — to a flat 3 bits via a fixed Walsh–Hadamard row rotation (CQ-3-rot) — pushing compression to ~4.8× while staying near-lossless (HellaSwag within ~0.005 of FP16 on Qwen2-0.5B/1.5B and Llama-3.2-1B; see the value-rotation section). It replaces the TurboQuant+ vector codec, which hit a similar ratio but with a 25% relative accuracy loss on grouped-query attention — while reusing that codec's Walsh–Hadamard value rotation, the one part of TurboQuant+ that always worked (origin + credit in the value-rotation section below).

Attribution — read this first

ChannelQuant is not a novel quantization algorithm. The scheme it implements is established prior art and we credit it without qualification:

• KIVI — Liu et al., ICML 2024. Per-channel INT4 for keys, per-token INT4 for values, motivated by fixed large-magnitude outlier channels in the keys. https://github.com/jy-yuan/KIVI
• KVQuant — Hooper et al., 2024. Per-channel-K + per-token-V + per-channel outlier isolation → near-lossless low-bit KV cache. https://arxiv.org/pdf/2401.18079

The contribution of this work is the silicon, not the math. KIVI and KVQuant are PyTorch/CUDA methods. There is, to our knowledge, no published streaming hardware codec that implements per-channel-key INT4 with calibrated, offline-fixed outlier isolation in RTL, integrated with an attention compute unit. ChannelQuant's novel pieces are exactly:

1. a residual-group buffer that makes per-channel-key scaling work in a streaming write path (you cannot scale by a column-max you have not seen yet — so the in-flight key group is held in FP16 until it fills, then quantized as a block);
2. a static outlier-channel ROM that turns KVQuant's runtime top-k detection into a per-layer mask lookup (valid because the outlier channels are a property of the trained weights, not the activation — see §c19 below);
3. the K/V-asymmetric datapath (buffered per-channel keys vs. streaming per-token values) and its synthesis on an open PDK.

Any claim of novelty in this repo or the paper is scoped to the hardware implementation and its evaluation. Overclaiming the algorithm is a hard failure of this project.

The evidence we build toward

HellaSwag acc_norm, n=250 (screening), Qwen2 GQA, quantizer applied uniformly to every K/V (no routing). Source: analysis/c17_{q05,q15,q7}_summary.json (reproduced from the predecessor APA c17 study; regenerated here in Phase 1). Values shown with Wilson 95% CIs in the source JSON.

4-bit variant	0.5B	1.5B	7B	verdict
FP16 (ref)	0.416	0.540	0.612	—
INT8 per-token (2×)	0.420	0.528	0.600	lossless 2× floor
naive INT4 per-token	0.372	0.248	0.212	collapses at scale
per-channel INT4	0.436	0.536	0.604	recovers ~FP16
KIVI (per-ch K / per-tok V)	0.408	0.540	0.600	recovers ~FP16
outlier (top-2 ch FP16 + per-ch)	0.428	0.552	0.616	best, ~FP16
MXFP4 (microscale FP4)	0.336	0.500	0.288	erratic — rejected
NVFP4 (microscale FP4)	0.384	0.412	0.380	erratic — rejected

The headline contrast — naive per-token INT4 collapses toward the 0.25 chance line at scale, while per-channel / KIVI / outlier recover ~FP16 — is large and robust. We do not over-claim per-model orderings among the recovering variants: at n=250 their CIs overlap (see §8 of REVAMP_SPEC.md). Headline paper numbers were re-run at n≥1000 in Phase 3 — the collapse-vs-recover contrast holds and CQ-4/CQ-4+ land within 0.02 of FP16 on Qwen2-0.5B/1.5B/7B (CLAIMS.md C12′). At the headline n, the FP16-outlier lane (CQ-4+) helps only marginally at D=128 (1.5B, non-significant p=0.09) and slightly hurts at D=64 (0.5B: 0.415 vs CQ-4's 0.421) — reversing the n=250 "outlier best" verdict in the 0.5B column above; CQ-4 is the default at every head dim (the outlier ROM/sidecar is optional). Regenerate: analysis/c23_headline.py --model Qwen/Qwen2-0.5B --tag q05 --n-items 1000 (+ q15).

Regenerate: MPLCONFIGDIR=/tmp/mpl /home/chaithu/lhs/.venv/bin/python analysis/c18b_quantizer_fig.py

Outlier-stability gate (c19) — PASSED at all scales. The static outlier ROM rests on key outlier channels being input-independent. Measured across 8 independent calibration batches, mean top-2 stability was 0.958 / 0.986 / 0.984 on Qwen2-{0.5B, 1.5B, 7B}; layer-0 (the noisiest layer) is perfectly pinned (1.00) at every scale; outlier-channel magnitude is 5.4–8.0× the median channel. Source: analysis/c19_{q05,q15,q7}_summary.json. → Build CQ-4+.

Tiers

Tier	Recipe	Combined ratio	Use
CQ-8	INT8 per-token (K and V)	~2×	safe lossless floor, all scales
CQ-4	per-channel-K / per-token-V INT4	~3.8×	the target, near-lossless
CQ-4+	CQ-4 + top-2 key outlier channels in FP16	~3.6×	best accuracy, GQA-heavy / 7B+
CQ-3-rot	CQ-4+ keys + WHT-rotated per-token INT3 values	~4.8×	flat 3-bit values, no calibration

No routing, no per-token adaptive bit allocation — the lever is granularity, not bit-allocation (predecessor c16). See REVAMP_SPEC.md §4 for the full compression accounting.

Value rotation → flat 3-bit values (CQ-3-rot)

Naive INT3 values collapse for the same reason naive INT4 does — one high-magnitude channel hijacks the per-token scale. Spinning each value row by a fixed Walsh-Hadamard transform before quantizing spreads that energy across all D dims, so no single channel sets the scale; unspin on read (the WHT is self-inverse). Values drop from 4 to a flat, uniform 3 bits with no per-model calibration, near-lossless on Qwen2-0.5B/1.5B and Llama-3.2-1B (HellaSwag within ~0.005 of FP16). Keys are untouched — the rotation only mixes a token's own dims, so it can't smear the per-channel key structure that sank TurboQuant+ (TurboQuant's value rotation was always fine; only its key rotation was the problem). Reference: compress_values_wht / decompress_values_wht in reference/channelquant_ref.py; the codec is bit-exact to the LonghornSilicon KVE hardware reference (wht_unit.sv, header-only channelquant_ref.hpp) on real Qwen tensors. It is add/sub only in hardware (D=64/128 are powers of two) and, being linear, composes with the P·V matmul so the inverse WHT runs once per decode step, not per cached token.

Provenance & credit. The Walsh–Hadamard value rotation is not original to ChannelQuant. It is the reusable "innocent half" of TurboQuant+ (themoddedcube/turboquant-plus — README §2 "Randomized Hadamard Transform (RHT) replaces QR rotation", turboquant/wht_kernel.py, docs/08_wht_kernel.md), where the Hadamard/RHT rotation was introduced by Chaithu Talasila — itself replacing the dense QR rotation of upstream TurboQuant (Zandieh et al., arXiv:2504.19874). ChannelQuant's own contribution is the reuse of only the value rotation — after establishing that the key rotation is what sank TurboQuant+ on GQA — paired with a calibration-free flat-INT3 tier (CQ-3-rot). Bringing that rotation to ChannelQuant's 4-bit-outlier problem was Abhiram Bandi's contribution — having read the TurboQuant+ paper, he identified its value rotation as the fix — developed here jointly with Chaithu Talasila (author of TurboQuant+ and its rotation).

Scope

ChannelQuant is the quantization algorithm + a bit-accurate reference model + accuracy/ablation experiments + the method paper. It is not the silicon. The RTL and tape-out live in the KVCE block (the kv-cache-engine repo — a silicon block of the Longhorn chip), which is being revamped separately to implement ChannelQuant. This repo's hardware-facing deliverable is docs/HW_CONTRACT.md (the algorithm-to-silicon interface KVCE implements against) plus golden test vectors (reference/testvectors/) for KVCE's Python↔C++↔SV parity. Silicon results (area/Fmax vs TurboQuant+) will come from the KVCE block and are noted as forthcoming — not produced here.

Repository layout

channelquant/
├── README.md                  # this file
├── NOTES.md                   # lab notebook — dated entries, full provenance
├── CLAIMS.md                  # claims ledger — every number → artifact + script
├── REVAMP_SPEC.md             # the design spec (source of truth)
├── analysis/                  # accuracy sweeps, calibration, group-size Pareto
│   ├── c17_quantizer_sweep.py        # the decisive 4-bit sweep (reproduced Phase 1)
│   ├── c19_outlier_stability.py      # static-mask gate (PASSED)
│   ├── c18b_quantizer_fig.py         # figure regen
│   ├── outlier_calibration.py        # offline per-layer outlier mask  (Phase 2)
│   ├── group_size_sweep.py           # G ∈ {32,64,128,256} Pareto      (Phase 2)
│   └── c1*_*_summary.json            # logged results
├── reference/                 # bit-accurate reference: CQ-8/CQ-4/CQ-4+/CQ-3-rot (Py, then C++)
│   └── testvectors/           # GOLDEN VECTORS for the KVCE block's 3-way parity
└── docs/
    ├── HW_CONTRACT.md                   # algorithm→silicon interface (KVCE consumes this)
    ├── research_kv_quant_landscape.md   # prior-art landscape (KIVI/KVQuant/vLLM)
    └── paper/                            # the method paper (maintained from Phase 1)

Build status

Phase	Deliverable	Status
0	Repo scaffold + README + NOTES + HW_CONTRACT	done
1	Reference model (CQ-8/4/4+) + reproduce c17 (±0.02) + golden vectors	done — c17 repro GATE PASS (Δ=0.000, screening); golden vectors confirmed, 9/9 ref tests pass
2	Outlier calibrator + group-size Pareto	done — knee G=128
3	Generalization (non-Qwen GQA + ARC-Challenge)	done — n=1000 headline + Mistral-7B / ARC-Challenge generalization (confirmed)
4	Method paper assembly	done — 4-page PDF (docs/paper/); optional venue polish pending
+	CQ-3-rot WHT value tier (post-Phase 4)	merged — bit-exact to KVE HW ref here; ~0.005 accuracy is screening, in-repo re-derive pending

Each phase gated the next (see REVAMP_SPEC.md §7) and all four have passed: the c19 static-mask gate, the Phase-1 c17 reproduction (Δ=0.000), the n=1000 acceptance gate, and generalization to a non-Qwen model + non-HellaSwag task. The CQ-3-rot WHT value tier landed afterward — bit-exact to the LonghornSilicon KVE hardware reference here (348,160/348,160 elems, 9/9 ref tests), but its ~0.005 accuracy delta was measured in the kv-cache-engine repo and is screening, pending in-repo re-derivation (see the CQ-3-rot rows in CLAIMS.md). Remaining work is the KVCE silicon companion (area/Fmax) and optional venue polish. RTL and synthesis are out of scope — they belong to the KVCE block, which implements this repo's docs/HW_CONTRACT.md and validates against reference/testvectors/.

Environment

export HF_HOME=/home/chaithu/lhs/.hf_cache        # ~/.cache is root-owned
export MPLCONFIGDIR=/tmp/mpl                       # for figure scripts
/home/chaithu/lhs/.venv/bin/python <script>        # transformers 5.10.2, torch 2.12.0+cu130

The predecessor KVCE pq4 vector codec hung the GB10; ChannelQuant is plain integer arithmetic (no PolarQuant tables), expected GPU-safe — but small/CPU validation runs first and all GPU runs are monitored.

Lineage

ChannelQuant supersedes TurboQuant+ (PolarQuant + QJL + Walsh–Hadamard rotation; repo themoddedcube/turboquant-plus, with history also carried in the kv-cache-engine repo), which reached a comparable ratio (~3.5×) but with a −0.10 HellaSwag acc_norm collapse on Qwen2 GQA. Note that ChannelQuant does not discard TurboQuant+ wholesale: it retains the Walsh–Hadamard value rotation from that codec (introduced there by Chaithu Talasila — see the CQ-3-rot value-rotation section) and drops only the key rotation, PolarQuant, and QJL. The diagnosis and decisive evidence originated in the adaptive-precision-attention lab notebook (studies c13–c19); ChannelQuant is a self-contained repo that re-derives its own artifacts. See docs/research_kv_quant_landscape.md for the prior-art landscape and REVAMP_SPEC.md for the full design.