[mache-cc1a70][mache-cc631a] Validate against an LLO candidate, and check the artifact not just the producer

[jamestexas]

Closes mache-cc1a70 and mache-cc631a. Together they make "validate mache against an ley-line-open release candidate, before either side ships" possible — which it was not.

1. The gated tests ignored the override production already honours

mache has exactly one leyline override, MACHE_LEYLINE_BINARY, which production's ResolveBinary checks before every pinned tier (#574). Its contract is already the right one: explicit opt-in, logs on use, fails loudly on a bad path, never a fallback.

internal/lltest didn't consult it. It called leyline.CachedPinnedBinary() directly, so it resolved the pin or skipped, regardless. Setting the override therefore pointed mache build at a candidate while TestLeylineSchema_MatchesPinnedBinary and every other pinned gate went on testing the pin — or skipped, since the conformance test additionally asserts pin == leylineSchemaVersion before re-deriving. The only sequence available was release → discover drift → fix, which is the sequence the pin exists to replace.

decideBinary now checks leyline.OverrideBinary first, exactly as production does. Unset, nothing changes: same cache resolution, same skip, still never downloads. Set, the resolver reports that binary's own version and flags the run as an override, so the conformance test turns its pin assertion into a report and states that a diff is the candidate's re-derivation worklist rather than drift.

A named-but-unusable override fails rather than skips. Skipping is right for the default path — no cached pin means there is nothing to test — but naming a binary states an intent, and silently testing nothing reports success for a validation that never ran. That choice is returned as data (decision.fatal) rather than made inline, because it is unobservable through a real *testing.T whose Fatalf terminates the test asserting it.

I got the design wrong first, and the gate caught it

My first version invented a second env var (MACHE_TEST_LEYLINE_BIN) and a second leyline-version parser. duplicate_definitions flagged both against the existing MACHE_LEYLINE_BINARY and extractSemver. It was right — reinventing the override would have left two knobs with two contracts for one job. Now leyline.OverrideBinary and leyline.ExtractSemver are exported and reused; "read leyline's version" has one implementation.

Also deleted lltest.PinnedBinaryOrSkip, an exported wrapper whose only remaining caller was in its own file. untested_function flagged it; deleting beat writing a test for a two-line delegation.

2. mache validated the producer, never the artifact

grep projection_schema_version returned zero hits. mache gated on the leyline binary's version and checked the artifact not at all, so a .db that was copied, cached, or built by somebody else's leyline was accepted unchecked.

The pin alone is demonstrably insufficient, and I confirmed that by building the thing rather than reading code: ley-line-open's projection-v4 branch reports leyline 0.18.2 (open) — byte-identical to the release whose schema it changes — so leylineVersionMatchesPin accepts it while it emits projection-v4. Filed upstream as ley-line-open-cbd3c9.

ProjectionVersion reads what the file declares; warnUnknownProjection runs at OpenSQLiteGraph as one indexed single-row lookup — constant cost, so Open's laziness guarantee (mache-b094fa) is undisturbed.

It warns rather than refuses. Every bump so far has been additive for readers — verified against a real v4 arena — and mache's readers already probe per column via ColumnExists. A hard failure would reject arenas mache can in fact read, including for library callers assembling many projections, where one newer shard would sink the corpus. When a projection appears that mache genuinely cannot read, that is the moment to refuse it by name, the way RequireProps refuses exactly the stale-props case.

Absence is the common case, checked rather than assumed: v0.18.0/v0.18.1/v0.18.2 carry no PROJECTION_SCHEMA_VERSION constant at all, so every released leyline writes an arena with no projection row, as does mache build. A row present but empty also reads as undeclared — the DDL is value TEXT NOT NULL, and NOT NULL is not non-empty. Mutation found that one: return v, true survived the first version of the test.

What this immediately found

Built ley-line-open's projection-v4 branch and ran both new gates against it. The complete re-derivation worklist, measured rather than asked for:

table	change
_ast	+ blob_ord INTEGER
nodes	parent_id TEXT → GENERATED ALWAYS AS (…) VIRTUAL
node_defs	+ node_kind, start_byte, end_byte, start_row, start_col, end_row, end_col
node_refs	+ the same seven

Nothing removed; _ast keeps every span column, so existing fallbacks stay valid. The occurrence columns carry the same names as _ast, so graph.RefRangeOf can read a span straight off node_refs instead of joining the 3.15M-row _ast table — which answers a question I had open with LLO without needing a reply.

The producer e2e reproduces mache-bc6ca3's failure exactly (cannot INSERT into generated column "parent_id"), and the generated expression the real binary emits is byte-identical to the one implemented in #633.

One baseline line, and why

god_file fires at count >= 10 AND count > 3 * mu. Adding a few small files lowered the mean enough to pull testdata/snapshots/medium-rust-rosary/src/dsse.rs — untouched, unchanged — over the bar. Grandfathered as a single line, consistent with the 24 sibling fixtures already in the baseline.

The underlying problem is filed as mache-f41b43: on main, 24 of 26 god_file findings are vendored testdata, so the fixture corpus is setting the threshold for mache's own code. fan_out_skew already excludes generated code for exactly this reason.

Verification

task test green (36 packages, TEST_RC=0), task smells 0. Both gates pass against the pin and produce the worklist above against the candidate. Every new assertion falsified by mutation.

🤖 Generated with Claude Code

https://claude.ai/code/session_01TtGhz7QzUHZi52a3FeNtEs