Decide: conformance-kit's vuln-gate vs the shared osv-scan lane — they now overlap

[bdelanghe]

conformance-kit ships gates/vuln-gate.mjs, which repos that vendor the kit run as their own dependency check. As of the org-wide rollout (#2), those repos also run the shared osv-scan lane. Two dependency gates, different databases, different postures.

How this surfaced

site#173 is red on conformance:

node vendor/conformance-kit/gates/vuln-gate.mjs
✗ vuln-gate: 3 known critical/high in all deps (0 critical, 3 high) · threshold 0

Pre-existing and unrelated to that PR — but it means the vendored gate has been there all along, and is currently failing. Worth recording against infra#104's premise that "no repo had any dependency scanning": true in effect, wrong in detail. That premise has now been corrected three times (the deno.lock gap, required-baseline.yml being inert, and this).

The actual differences

	vuln-gate.mjs	osv-scan lane
source	npm audit-shaped	OSV database
scope	vendored per repo, npm only	every ecosystem osv-scanner supports + deno.lock npm subset
threshold	$VULN_THRESHOLD, counts critical/high	any known vulnerability
posture	hard-fail at threshold	hard-fail, with report-only adoption grace
bump path	re-vendor the kit per repo	one pin edit, fleet-wide

Redundancy here is not automatically bad — the two consult different advisory sources, and a finding in one that the other misses is real signal. The problem is that nobody has decided, so repos will drift into having two gates with different thresholds and no stated relationship.

Options

• Coexist, documented. Keep both, write down which is authoritative for merge-blocking and which is advisory. Cheapest; leaves two things to maintain.
• Shared lane supersedes. Remove vuln-gate from the kit, point vendoring repos at osv-scan. One definition, but drops npm-audit coverage that OSV may not duplicate.
• vuln-gate becomes an osv-scan caller. The kit stops implementing scanning and just calls the shared workflow. Preserves the kit's interface, single implementation underneath.

Done when

• A decision is recorded and the losing gate is removed or explicitly marked advisory
• Count how many repos actually vendor conformance-kit (unknown — needs an org-wide code search, which a repo-scoped agent session can't run)
• site's 3 pre-existing highs are cleared or accepted with a written reason (separate from this decision — see the remediation queue)

[bdelanghe]

A third scanner: repo-standard.yml already runs OSV + dependency-review

hooksmith's adoption PR (#102) came back with these checks:

success  osv / osv-scan                  <- the lane this rollout adds
success  standard / osv                  <- repo-standard.yml
success  standard / dependency-review    <- repo-standard.yml
success  standard / test

So repos that adopted the opt-in bounded-systems/.github/.github/workflows/repo-standard.yml caller have had OSV and dependency-review running all along. That's distinct from required-baseline.yml, which advertises the same coverage org-wide and has never run — repo-standard is opt-in and actually executes.

This is now the fourth correction to "no repo had any dependency scanning"

#	what was found	where
1	deno.lock unscannable — Deno-first repos scanned nothing	#1
2	required-baseline.yml authored but never injected	infra#135
3	conformance-kit ships a vendored vuln-gate.mjs	this issue
4	repo-standard.yml runs OSV + dependency-review for its adopters	here

infra#104's premise held in effect — nothing was reliably scanning everything — but the org had three partial mechanisms already, and each one turned up only when a lane happened to run beside it.

What this changes for the decision

The overlap question is no longer two-way. A repo can now be running up to three dependency scanners:

	source	posture	coverage
osv-scan (this repo)	OSV	hard-fail, grace during adoption	all osv-scanner ecosystems + deno.lock npm subset
standard / osv	OSV via google/osv-scanner-action	per repo-standard	whatever that action defaults to
vuln-gate.mjs	npm-audit-shaped	threshold-based	npm only, vendored copy

Two of those query the same database. That's not redundancy that buys anything — it's the same advisory reported twice with different failure semantics, and a maintainer seeing standard / osv green next to osv / osv-scan red (or vice versa) has no stated way to know which to believe.

Suggested reframing of this issue: rather than "vuln-gate vs osv-scan", decide the org's whole dependency-scanning topology — which single lane is authoritative, and what happens to the other two. Candidate: osv-scan becomes the one implementation; repo-standard drops its osv job and calls it; vuln-gate either retires or narrows to whatever npm-audit finds that OSV genuinely doesn't.

Also worth checking: how many repos run repo-standard, and whether its osv job would have caught the conformance-kit findings in the repos that vendor the kit.

---

Generated by Claude Code