Two merge gates that decide from a report, not an exit code - #7
Merged
Conversation
…ather than an exit code Semgrep, and the licence half of dependency review. Both run on Linux in their own workflow, because neither needs anything built - which is the split ci.yml already named as the one worth making. Neither gate trusts the tool's exit code, and that is measured rather than stylistic. `--severity ERROR` is not a filter that works: the flag knows only INFO, WARNING and ERROR while registry rules also carry HIGH and CRITICAL, so it silently ignores exactly the severities it is asked to block. And `semgrep scan` exits ZERO whatever it finds - including when pointed at a config that does not exist, where it also writes the JSON file it was asked for and fills it with nothing. Every signal a caller would normally trust then says the code is clean. So the gate reads the report and blocks on ERROR, HIGH or CRITICAL, on a scan error, and on a report that read no files. That last one closes the case above: measured here, a bad config produced zero results, zero errors and an empty list of scanned paths, and the file count is now printed on every run so a collapse from hundreds to a handful is visible rather than quiet. Worth stating plainly: this gate is not decoration. Against this tree `p/default` reports nothing, and against the two release workflows as they were before 3b94c4e it reports six findings, every one ERROR and every one the shell injection that PR removed. It also means that invariant now has an enforcer, which it did not have when it was written. The dependency half is two steps. The action blocks a dependency added with a known vulnerability, with `comment-summary-in-pr: never` written out rather than left to a default that belongs to somebody else - nothing is posted onto a pull request, the log has it and a red check is the message. The script does what the action documents that it will not: an undetermined licence blocks. For a GPL-3.0 project shipping a binary, "we could not tell" is the one answer nobody can act on. Its allowed list is read out of deny.toml and repeated nowhere, so this gate and `cargo deny check licenses` cannot come to disagree - this repository has paid for that kind of drift before. A consequence, stated because it will surprise somebody: that list is short, so a perfectly GPL-compatible licence which is simply not used here yet will block. Blocking means a person decides. Actions are skipped in the licence half. GitHub reports no licence for any of them, they are CI machinery that reaches no user, and a gate that fires on every pull request touching a workflow is a gate people learn to bypass. What actions are held to instead is stricter and lives elsewhere: a full commit SHA, moved by Dependabot together with its version comment. Both scripts were watched failing. The semgrep gate on findings, on a scan that read no files, and on a missing report. The dependency gate across thirteen cases: an unknown licence, an empty one, a denied one, one half of an OR that is not allowed, a removed dependency it must ignore, an action it must skip, and a deny.toml it cannot read. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two gates that block a merge: a static-analysis scan, and the licence half of dependency review. Both run on Linux in their own workflow, because neither needs anything built - which is the split
ci.ymlalready named as the one worth making.Neither gate trusts an exit code, and that is measured
--severity ERRORis not a filter that works. The flag knows onlyINFO,WARNINGandERROR, while rules from the registry also carryHIGHandCRITICAL, so it silently ignores exactly the severities it is asked to block.Worse, and measured here on 2026-09-08:
semgrep scanexits zero whatever it finds - including when pointed at a config that does not exist, where it still writes the JSON file it was asked for and fills it with zero results, zero errors and an empty list of scanned paths. Every signal a caller would normally trust then says the code is clean.So the verdict is made by a script reading the report. It blocks on
ERROR,HIGHorCRITICAL, on a scan error, and on a report that read no files. The file count is printed on every run, so a collapse from hundreds to a handful is visible rather than quiet.Is it worth anything
Measured both ways, which is the only way to answer that.
3b94c4eERRORThose six are the shell injection removed in #6. That invariant - no
${{ }}inside arun:block - now has an enforcer, which it did not have when it was written.The dependency half
Two steps, because the two halves are different questions.
The action blocks a dependency added with a known vulnerability.
comment-summary-in-pr: neveris written out rather than left to a default that belongs to somebody else. Nothing is posted onto a pull request - the log has it, and a red check is the message.The script does what the action documents that it will not: an undetermined licence blocks. For a GPL-3.0 project that ships a binary, "we could not tell" is the one answer nobody can act on.
Its allowed list is read out of
deny.tomland repeated nowhere, so this gate andcargo deny check licensescannot come to disagree. One consequence, stated because it will surprise somebody: that list is short, so a perfectly compatible licence which is simply not used here yet will block. Blocking means a person decides, not that the answer is no. Widening it means editingdeny.toml, which is the same act the dependency rule asks for anyway.Actions are skipped in the licence half. GitHub reports no licence for any of them, they are CI machinery that reaches no user, and a gate that fires on every pull request touching a workflow is a gate people learn to bypass. What actions are held to instead is stricter and lives elsewhere: a full commit SHA, moved together with its version comment.
Watched failing
The scan gate: on findings, on a scan that read no files, and on a missing report.
The dependency gate, across thirteen cases: an unknown licence, an empty one, a denied one, one half of an
ORthat is not allowed, a removed dependency it must ignore, an action it must skip, and adeny.tomlit cannot read.After merging
These two checks block nothing until their contexts are added to the branch ruleset, next to
Gatesandsubmit-nuget. That is a repository setting rather than a file in this diff.🤖 Generated with Claude Code