fix: scan the release image before its version tag is bound [#5] - #20
Merged
Conversation
image-build pushes by digest and image-publish binds the tag; nothing in between looked at what was being published. The PR-time Security Scan does not cover it — it grades an image built from the PR's tree on the day the PR ran, while a tag cut weeks later is a fresh build off a rolling wolfi-base and a freshly resolved module graph. trivy-release-gate was removed earlier in this branch as dead code, correctly at the time: its only caller was build-and-push.yml. Adding ci-release.yml put a release path back without putting the gate back with it. Reinstated against the pushed digest rather than a local tag, since push-by-digest leaves no local reference. The digest is unreachable by name until image-publish binds a tag, and the gate runs before that, so a finding means the version tag is never created. --exit-code 1 with a table: there is no PR to comment on, so the findings belong in the log the red check points at. Scanned per arch on its own native runner. Also adds timeout-minutes to all four jobs. The default is 360, and the new gate pulls from a registry — a stalled pull would otherwise hold a runner for six hours, and on a release leg it blocks Publish Artifact behind `needs`, so the tag simply never appears with no failure to point at.
|
📊 Test Coverage: ➖ Not applicable — no non-test Go files changed vs |
🛡️ Trivy security scan (CRITICAL,HIGH,MEDIUM,LOW)Go dependenciesNo findings at CRITICAL,HIGH,MEDIUM,LOW. Container imageNo findings at CRITICAL,HIGH,MEDIUM,LOW. |
] cover-diff has two paths that compute no percentage — no changed non-test .go files, so there is no denominator, and changed files carrying no coverable statements, which would be 0/0 — and both reported "Test Coverage: ✅ Passed — not applicable". Read from the PR that is a green pass, so a CI-only or docs-only PR looked indistinguishable from a well-tested one, and there was no number to reconcile the claim against. Both now report "➖ Not applicable" and say which case applies and why there is no percentage. Still exit 0 — a PR that changes no Go code must not be blocked on Go coverage — so the check verdict is unchanged; only the claim the comment makes about it is. Also corrects "no changed Go files" to "no non-test Go files changed": the changed-file set already filters _test.go, so a test-only PR took this path while the message said no Go files had changed at all.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #14. Found while reviewing discovery-service#15, which does this correctly and this repo did not.
The gap
ci-release.ymlpublishes an image that nothing has ever scanned.image-buildpushes by digest,image-publishbinds the version tag, and there was no scan between them. The PR-timeSecurity Scandoes not cover it: it grades an image built from the PR's tree on the day the PR ran. A tag cut weeks later is a different build — freshly pulledcgr.dev/chainguard/wolfi-base(a rolling tag) and a freshly resolved module graph. The artifact that ships was not the artifact anything looked at.How it got here, since it's worth recording:
trivy-release-gatewas removed early in #14 as dead code, which was correct at the time — its only caller was the deletedbuild-and-push.yml.ci-release.ymlwas added later in the same branch and put a release path back without putting the gate back with it.The fix
make trivy-release-gate ARCH=<arch>, betweenimage-buildandupload-artifact:push-by-digestleaves no local reference, sodigest-<arch>.txtis the only handle on the layers just built.image-publishbinds a tag, and the gate runs before it — a finding means the version tag is never created.publish-artifactneeds both legs, so either arch failing is enough.--exit-code 1and a table, not SARIF at--exit-code 0. There is no PR to comment on, so the findings belong in the log the red check points at, and the scan is the gate rather than a report something else grades.Same
SEVERITY(CRITICAL,HIGH,MEDIUM,LOW) and the sametrivy-cachecomposite action as the PR scan, so the release band can't drift from the PR band.Also:
timeout-minuteson all four jobsEnd to End verificationSecurity ScanBuild Artifact (<arch>)Publish ArtifactThe default is 360 minutes. This matters more now that a release leg pulls from a registry: a stalled pull would hold a runner for six hours, and because
publish-artifactsits behindneeds, the tag would simply never appear with no failure to point at. Generous multiples of the real durations — this is a hang detector, not a performance gate that goes red when the suite legitimately grows.Checks
No change to the check set. Still
CI / End to End verification+CI / Security Scanon a PR,CI / Build Artifact (amd64),CI / Build Artifact (arm64)andCI / Publish Artifacton a tag. Branch protection needs no further changes.Verification
actionlintclean.trivy-release-gateexercised across all five paths with a stubbed trivy:digest-<arch>.txtmissing::error::, trivy never invokeddigest-<arch>.txtemptyghcr.io/openagrinet/network-adapter@sha256:…— correct ref, lowercasedGITHUB_REPOSITORYunsetrequire-image-repoEnd-to-end tag run linked in a comment below.
Also in this PR: coverage reported "Passed" when it measured nothing
Raised on review of this PR's own coverage comment: "what is the percentage? how do we say as passed?" — a fair question, because there is no percentage and there never was.
cover-diffhas two paths that compute nothing:.gofiles — no denominator0/0Both printed
📊 **Test Coverage: ✅ Passed** — not applicable. From the PR that reads as a green pass, so a CI-only PR was indistinguishable from a well-tested one, and there was no number to reconcile the claim against. Now:Three verdicts instead of two:
✅ Passedand❌ Failedboth now mean a percentage was computed and compared againstMIN_COVERAGE. Exit status is unchanged (still0) — a PR that changes no Go code must not be blocked on Go coverage — so the check verdict is the same; only the claim the comment makes about it changed.Also corrects "no changed Go files" to "no non-test Go files changed": the changed-file set already filters
_test.go, so a test-only PR took this path while the message said no Go files had changed at all.Tag verification
Tag
v0.0.0-citest3cut on the head commit, run 34154000953 — all three jobs green. The gate is doing real work on both arches, not passing vacuously:0 findings at
CRITICAL,HIGH,MEDIUM,LOW, so both legs passed andPublish Artifactbound the tag to a proper multi-arch index:The trivy cache missed on this run, as expected — the
Makefileedit changedhashFiles('Makefile'), which is what that key is for. The log confirms it re-fetched from the pinned.../trivy/v0.74.0/contrib/install.sh.Git tag deleted. The GHCR package version
v0.0.0-citest3needs manual deletion — my token lacks the scope.