M2X-041: make the Phase 1B gate worth running — seal, replayable fixtures, derived citations

[yashpancholi09]

Makes the Phase 1B gate worth running. Three fixes, one per commit.

Run before M2X-040 rather than after it. This ticket says "ONLY if M2X-040 fails" — which, followed literally, means burning the sealed held-out set to unlock the ticket that repairs the thing the run would have failed on. Supervisor's decision on 2026-08-13: fix first, gate later. Recorded in docs/design/day4-gate-recovery.md, docs/gates.md and on the Odoo tickets.

1. The seal did not exist (4a1524a)

git ls-files eval/labels/heldout/ returned .gitkeep and nothing else, against a CLAUDE.md rule that says the seal is physical.

Two artefacts, because they answer different questions. The ciphertext makes the set recoverable. The SHA-256 manifest makes it checkable — and it is the one that proves anything, since gpg salts every invocation and a .gpg diff therefore cannot distinguish a re-seal from a content change. Digests leak nothing, so verify runs without the passphrase: a reviewer can confirm the set is unedited while remaining unable to read it.

gpg rather than age, per the supervisor — age is not installed here. The *.age whitelist stays.

Recorded, not glossed: the seal was applied after the cases were written and after dev iteration had run, so it makes edits visible from this commit forward and says nothing about the window before it.

2. The number was not reproducible by anyone (99874f0)

v3 scored 0.4279 on one checkout and 0.3086 on another — same prompt, matcher, threshold and 15/15 case set, differing only in which sampled outputs each git-ignored data/cache/ held.

--fixtures replay scores committed outcomes and contacts no provider. Verified end to end on the real dev set: record and replay both print 0.3882, with identical per-field TP/FP/FN and the same schema failure named.

refusal	what it prevents
missing fixture aborts	a case leaving the denominator invisibly
transcript-digest mismatch aborts	scoring an old answer against edited eval/tiron/ words
provider failures never recorded	a 429 becoming a permanent gate number

Schema failures are recorded — 100% schema validity is a gate leg, and a fixture set of only successes would report it green by construction.

Honest caveat: replay removes the generation provider, not the embedding one. Embeddings are a single forward pass with no sampling and reproduce given the same model; generation samples. Freezing the sampled half is sufficient.

3. Citation drift (6ed6794)

The extractor paired a segment id with the previous line's timestamps (seg-0033 cited as 580.3-581.4 when it runs 581.44-586.445), burning the case's whole retry budget. v3's explicit citation rule did not move it.

Evidence.t_start/t_end are now derived from segment_id; any value the model supplies is discarded. segment_id still resolves, and that is the check that catches invention. Gives up sub-span citation deliberately — all 84 cited items in the dev labels name a whole turn.

v7 was written to test the prompt half, measured, and lost. Like-for-like over the fourteen cases both prompts answered: v3 0.3882, v7 0.3704. Its 15/15 schema-validity is not a demonstrated fix either — v3 also scored 15/15 at d486372, so the single failure here is sampling noise. v3 stays pinned; v7 stays on disk with a changelog row saying it failed.

Nothing is lost by not pinning it, because the fix is in the schema. Instructor renders Field(description=...) into the request, so "Do not emit this field" reached the model and overrode v3's own instruction — every citation on the v3 run was {"segment_id": ...} and nothing else.

Injections re-measured under the new contract: still 3/3.

Review notes

• Four existing tests were changed, not added — they asserted the contract this replaces. Two rejections are now unreachable rather than caught, which is the stronger property.
• Not in this branch: M2X-041's fresh 5-case held-out retry. Those labels must be written after the fixes settle or they target a moving object; the sealing tooling they need is what this delivers.
• One manual step remains — the ciphertext needs the Evaluator's passphrase, deliberately unavailable to the tooling that generated the manifest: uv run python scripts/seal_heldout.py seal.

566 tests green.