H20 heuristics on the bounded-candidates flow + measured-config autotune

[guzekai01]

Stacked on #59 (which stacks on #58), rebased onto its latest head (e2aa2f3, the policies/seed-selection revision): the leading commits in the diff belong to those PRs and will drop out once they merge; this PR's own changes are the five feat(tune): commits.

What

Builds on #59's bounded schedule candidates flow (problem → bounded candidates → shared analysis → explicit selection → audit trail) in three
directions:

1. H20 heuristics in terminal form on the candidate flow. Following
   this PR's base revision ("policies own seed selection"), the former H20
   legacy get_config body now lives in the policy file as
   build_h20_seed_config — the single seed implementation for every
   dtype/construction — and sm90_h20.py is a thin dispatcher; the move
   was verified by a 1644-point pre/post snapshot (both grids plus the
   out-of-scope dtype universes): zero drift. ** The fused-E8M0 8-bit
   universe — MoE (all tile sizes, including the long-K small-tile residency
   tuner) and dense (plain / TMA warp-spec / deep-pipeline arms) — now
   selects through ScheduleCandidate/CandidateAnalysis/TuningDecision.
   Selection reproduces the legacy heuristic bit-for-bit (guarded by
   ~1.8k-point parity grids); what changes is that every decision now
   records which alternatives were considered and why one was preferred.
2. Register-budget analysis. CandidateAnalysis gains
   register_budget_per_thread (launch-bounds model, 8-aligned, 255 cap)
   and register_pressured (against a family-supplied measured demand).
   This institutionalizes the H20 finding that motivated the earlier CTA
   clamp: at 128 threads x 3 CTAs/SM the budget (168) sits below the
   measured demand (~236) of grouped-input-scale mainloops, forcing
   local-memory spills (measured 1.4x latency).
3. Measured-config autotune. Offline: python -m humming.tune sweeps
   the bounded search space per shape_m with a two-stage measurement
   (coarse → top-k fine) gated by a golden-output correctness check, and
   saves an m-bucket table keyed by layer+gemm+flags and fingerprinted by
   gpu/sm/num_sms/humming/cuda versions. Runtime: get_tuning_config()
   serves the measured winner on a valid cache hit and falls back to the
   heuristics otherwise. The search space itself is analysis-driven: shared
   candidate analysis prunes provably-illegal configs before any benching
   (every pruning rule is backed by a kernel static_assert), which removed
   ~82% of the compile-failing candidates a sweep used to burn time on.

Usage

Offline tuning (per layer/gemm; writes ~/.humming/tune_cache/):

python -m humming.tune \
  --shape_n 6144 --shape_k 7168 --num_experts 48 --top_k 8 \
  --a_dtype float8e4m3 --b_dtype float4e2m1 --bs_dtype float8e8m0 \
  --c_dtype bfloat16 --weight_scale_group_size 32 \
  --input_scale_group_size 128 --gemm_type indexed --fast

Sweep + report (reusable evidence/regression tool):

python tools/tune_sweep.py --out-dir /tmp/tune_sweep --fast
# per-m table: heuristic ms vs tuned ms vs speedup, plus win/tie counts

Runtime:

from humming.tune import get_tuning_config
config = get_tuning_config(layer_config, shape_m, gemm_type="indexed")
# measured winner on a valid cache hit; heuristics otherwise.
# HUMMING_USE_MEASURED=0 disables cache lookups globally.

Audit ("why this config?"):

from humming.tune import explain_tuning_config
decision = explain_tuning_config(layer_config, shape_m, gemm_type="indexed")
# TuningDecision: selected + all considered candidates with rejection
# reasons + a human-readable reason. For measured hits the reason carries
# the cache key, creation timestamp and m-bucket.

Effect (H20, fused-E8M0 fp8xfp4, DSV4-Pro TP8 shapes, m swept to 8192)

Produced by tools/tune_sweep.py (fast space, strict golden gate); the
measurement gate never selects a config slower than the heuristic baseline,
so the floor is a tie. Shapes are the moe_tp=8/ep=1 per-rank projections of
DSV4-Pro (hidden 7168, moe_intermediate 6144, E=384, top_k=8).

workload	shape	wins / total	geomean(win)	best	notes
dense w13	1536x7168	11/23	1.98x	2.40x	small m (<=192) already optimal -> honest ties; m>=256 all wins 1.7-2.4x
dense w2	7168x768	23/23	1.61x	2.56x	short-K dense is a weak heuristic domain; large-m plateau 2.0-2.2x
MoE w13 (indexed)	1536x7168, E=384	20/20	1.19x	2.14x	5-stage small-tile winners
MoE w2 (indexed)	7168x768, E=384	8/20	1.04x	1.18x	short-K MoE heuristic near-optimal; rest tie
dense (large-shape ref)	6144x7168	23/23	1.46x	2.20x	full w13 shape kept as a large-shape reference

Zero sub-1.0x rows and zero missing-baseline rows across all sweeps. Tune
winners are stable across the terminal-form refactor (flips only between
near-tied candidates, speedup drift within 3%).

Validation

• 130 tune/policy tests on an H20 pod plus the full repo suite (456
  passed / 86 device-skipped; the 33 failures are pre-existing hadamard
  suite failures reproduced identically on this PR's base). Grids for the
  three families (selector output == seed heuristic on every in-scope
  point; negative boundaries served by the seed), Refactor SM90 tuning around bounded schedule candidates #59's own suites, and
  the tune pipeline suites (cache round-trip/corruption, measurement
  orchestration incl. worker death and correctness-gate semantics, CLI
  validation).
• Grids and selection arms are mutation-checked: breaking any arm or seed
  branch turns the corresponding grid red.
• The terminal-form move was gated by a 1644-point pre/post output
  snapshot: zero drift, including the out-of-scope dtype universes.
• E2E on H20: full CLI tune runs produce fingerprinted tables; runtime
  lookup serves the measured winner; cache misses and stale fingerprints
  fall back to heuristics.

A finding from the sweep: stream-K tail-error vs the golden gate

While validating the sweep we found that stream-K schedules fail the golden
gate on ~0.002-0.004% of output elements (individual absolute errors of 2-3;
the remaining 99.996% sit inside quantization noise; the same schedule with
use_stream_k=False has zero violations, on both dense and indexed). The gate
is an all-element assert (rtol=0.05/atol=0.5), so every stream-K candidate —
including the stream-K heuristic baseline — is rejected, and tuned tables
select from the non-stream-K space only (which still beats the heuristic
substantially, see the tables above). This is a pre-existing kernel-side
numeric behavior, not introduced here. Kept as-is for this PR (conservative
gate); follow-ups tracked separately: (a) root-cause the stream-K reduction
tail, (b) discuss a statistical gate (violation-fraction + percentile bound)
that matches e2e accuracy sensitivity.

Not in this PR (planned follow-ups)

• Migrating the remaining H20 families (16-bit activations, non-fused-E8M0
  MoE) onto the candidate flow with their own parity grids.
• A rule for ptxas per-construction register floors (the residual ~4% of
  compile-failing sweep candidates; absorbed by the poison machinery today).
• Feeding the measured selector directly from family candidate enumeration.
• Cache invalidation granularity. Today a tune table is invalidated by
  exact humming_version match (every commit), and the JIT cache keys on
  .cuh mtimes (any reinstall/image rebuild). Planned: (a) switch both to
  content hashes of the sources that actually participate (the generated
  kernel source is already part of the JIT key, so only the .cuh mtime
  component needs replacing), so no-op updates reuse caches; (b) a
  re-validation fast path for tune tables — on fingerprint mismatch, the
  old winner is re-benched against the heuristic behind the same golden
  correctness gate (incl. bucket-interior checks) instead of being
  discarded: it passes the identical verification a full re-tune would
  apply, trading only optimality (marked validated-stale, refreshed by a
  background re-tune) and never correctness.
• SGLang integration: wiring get_tuning_config into SGLang's humming
  MoE runner so serving picks up tuned tables (env-gated, heuristic
  fallback), plus a deployment story for shipping tune caches with images.
  We run this stack in production-like H20 serving and will follow up with
  the SGLang-side PR.