Re-land payout-artifact reuse with event-driven validity and a kill-switch

[djh58]

Problem

Tip-change refresh latency is dominated by the shared bundle build, and profiling attributes the build almost entirely to the synchronous reward-window walk — a multi-second ledger read that runs far more often than payout state actually changes. Every unnecessary walk extends the window in which miners grind a stale tip (forfeiting block solves as tip_moved abandons) and queues initial-job builds behind the single-flight future, which is what starves the stratum health probes.

#102 and #103 already built the reuse fix and were both emergency-rolled-back (#104 abandoned the line). Re-reading both incidents against production cadences shows they were the same arithmetic error at different anchors: a wall-clock validity budget shorter than the interval between consecutive shared builds, applied to an object whose rebuild pays a multi-second walk.

• Round 1 (Make payout-ledger-artifact reuse anchor-scoped so it survives production share rates #102): freshness measured from the pre-walk anchor → every slow build arrived born-expired → continuous re-walk under the prepare lock.
• Round 2 (Survive production build latency in payout-artifact reuse (2026-07-29 re-land) #103): freshness moved to install time → the budget expired between consecutive builds → every build fell back to the synchronous walk while the re-arm worker walked the same window in parallel → delivery brownout.

Correctness never needed the wall-clock knob: the payout-generation and balances-sha fences invalidate on real payout events, and post-anchor shares settle in the next window by construction (the anchor contract #102 documented).

Change

Four commits on top of 1.x.x:

1. Revert Revert the anchor-scoped payout-artifact reuse line (#103, #102) #104 — a mechanical git revert of the abandonment commit (~1,800 lines, zero hand-edited). It restores the reuse line to the exact tree previously reviewed and merged as Survive production build latency in payout-artifact reuse (2026-07-29 re-land) #103 — which already included Make payout-ledger-artifact reuse anchor-scoped so it survives production share rates #102 and the round-3 hardening from artifact-reuse-latency-hardening (install-site freshness stamps, probe restamp admission, publication born-expired gate, event counters).
2. Event-driven validity below the audit ceiling — the probe serves any armed artifact whose generation/difficulty/balances fences hold and whose declared anchor is inside PRISM_PAYOUT_ARTIFACT_MAX_ANCHOR_AGE_SECONDS. Past the new PRISM_PAYOUT_ARTIFACT_REANCHOR_SECONDS floor it schedules the debounced background re-anchor and keeps serving. PRISM_PAYOUT_ARTIFACT_REUSE_STALENESS_SECONDS is removed; cached no-artifact bundles gate on the re-anchor floor (their declared anchor is already ceiling-gated, and the bundle-cache TTL bounds them tighter in production). PRISM_PAYOUT_ARTIFACT_REUSE (default on) is a master kill-switch: off disables the reuse line — probe refusals, no background walks, no arming — restoring pre-reuse delivery economics, while the synchronous build path keeps the re-landed anchor-selection semantics. Both prior incidents required emergency redeploys; a third delivery regression is an env flip.
3. Observability — served_reuse/probe_rejected_ceiling join the payout-artifact event family; a semantic_current_work_ratio gauge (fingerprint + payout generation) lands beside the strict object-identity coverage gauge (which reads 0.0 essentially continuously under generation churn, but stays untouched because it drives mining-health gating); waves >1s log one attributable line.
4. Phase-counter blind-spot fix — the job-build executor and artifact prep workers now flush their thread-local phase accruals; the wave-level phases (template, ledger, assembly, bundle) exported exactly 0.0 despite continuous fetches and multi-second walks because those threads never passed through the per-client flush points.
5. Adversarial-review hardening — an independent adversarial review of this PR confirmed the revert purity, the round-2 diagnosis, serve-window correctness, and the zero-regression claim, and refuted the original "restores exactly" kill-switch wording. Its findings are folded in: startup validation refusing an anchor ceiling at or below 2x the re-anchor floor (one env var could previously re-create the round-2 walk-per-build shape mechanically), the arming gates above, a memoized balances digest replacing per-probe O(accounts) canonicalization, an idle-issuance ceiling gate (the one serving route that bypassed the audit ceiling), and a probe_past_floor canary event so serving-past-floor is observable independently of rearm_scheduled.

Why re-land via revert instead of fresh commits. Commit 1 being a pure revert means the reuse base is byte-identical to code that was already reviewed twice and hardened through three review rounds — reviewers can verify zero drift by diffing commit 1's tree against the #103 merge commit (cd32c13); the only differences are the unrelated #97/#106 changes that landed after #104. The genuine review surface of this PR is therefore commits 2–4: the validity-semantics delta plus the observability additions — exactly the difference between "what browned out" and "what this ships." One consequence to be explicit about: commit 1 alone reintroduces the browned-out behavior; the PR is atomic, and commit 2's event-driven semantics plus the kill-switch are what make the third landing safe. (A squash merge collapses this distinction, but the commit boundaries are what make the review tractable.)

Tests

tests/test_prism_coordinator_job_cache.py: 174/174. Install-age rejection tests became serve-past-install-age brownout regressions; rearm/debounce and landed-preview suppression age through the anchor; in-flight retention ages through the ceiling; the kill-switch has dedicated coverage (probe refusal + zero background scheduling + clean re-enable).

Full suite: 1,390 tests — failure set byte-identical to clean 1.x.x in the same environment (30 pre-existing sandbox failures — 27 Makefile-lifecycle errors from the missing make binary plus 3 deploy-pin failures; zero regressions). cargo test -p qbit-prism not run locally (no toolchain in the build sandbox; no crate files touched) — CI covers it.

Deploy notes

• Interacts favorably with two pending infra changes: raising the template-cache TTL (cuts the same-tip churn that triggered round 2) and deprioritizing the periodic backup's disk IO (reduces the cold-cache reads the reward-window walk pays). Neither is required for safety — the semantics no longer interact with build latency at all — but both shrink the residual.
• Rollback: PRISM_PAYOUT_ARTIFACT_REUSE=0 + restart. No redeploy.
• Tripwire: qbit_prism_stale_job_abandons_total{class="balance_stale"} must stay 0.

Expected outcome (verifiable within hours of deploy)

served_reuse dominating the event family with full reads tracking the payout-event rate; ledger_snapshot durations collapsing between payout events; tip→refresh latency dropping to low single-digit seconds except immediately after pool-won blocks (until a follow-up speculative pre-build); tip_moved abandons and health-probe timeouts falling toward their propagation-and-noise floors.

🤖 Generated with Claude Code