feat(analyst): ship public real-model benchmark loop

[drewstone]

Summary

• adds the public agent-eval analyst-benchmark command for pinned AgentRx and CodeTraceBench inputs
• makes clusterId and labelState explicit and removes inferred label semantics
• persists paid responses, cost records, and hash-chained observations for exact continuation and fail-closed accounting
• publishes two 64-call GLM-5.2 Agent Eval runs plus one pinned CodeTracer comparison on the same 32 trajectories
• modularizes scoring, summaries, public adapters, result validation, persistence, calibration, and response caching

Measured results

Arm	Valid	Official all-row F1	Calls	Same-rate cost
Agent Eval + final tests	63/64	13.47%	64	$1.208462
Agent Eval trajectory only	62/64	15.02%	64	$1.162426
CodeTracer trajectory only	61/64	11.61%	933 known	$7.270347 over 63 known runs

The result is deliberately failure-inclusive. Precision, repeat agreement, and solved-case false positives remain weak. The 32-case mini-SWE subset is pinned and reproducible but is not representative of all CodeTraceBench.

Proof

• 4,576 tests passed, 3 skipped across 341 files
• TypeScript source and examples compile
• Biome checked 646 files
• build and packed-package verification passed
• pnpm audit: 0 known vulnerabilities
• Semgrep: 492 rules over 664 files, 0 unsuppressed findings in changed files
• exact live-key scan: 0 matches across 71 changed files
• both completed GLM runs resumed without model calls in 0.38s and 0.33s with byte-identical outputs
• input check: 32 traces, 64 final-test artifacts, 0 missing cases

Breaking change

AnalystBenchmarkCase now requires clusterId and labelState. No compatibility inference or shim is included.

[tangletools]

✅ Auto-approved drewstone PR — 04c856ab

This PR was opened by the trusted drewstone account.
The full PR reviewer audit still runs separately and will publish findings if it detects issues.

_{tangletools · auto-approval · reason: drewstone_author · 2026-07-30T15:57:38Z}