Skip to content

fix: verify release artifacts before publishing and make re-runs safe - #116

Draft
hllvc wants to merge 2 commits into
mainfrom
fix/release
Draft

fix: verify release artifacts before publishing and make re-runs safe#116
hllvc wants to merge 2 commits into
mainfrom
fix/release

Conversation

@hllvc

@hllvc hllvc commented Aug 31, 2026

Copy link
Copy Markdown
Collaborator

Summary

Reworks the provider release pipeline so a GitHub release is only published after every artifact has been downloaded back and verified, and so a failed release can simply be re-run. Also unpins GoReleaser from the end-of-life v1 line, moves the workflows onto Node 24 runtimes, and expands the build matrix from 6 platforms to 13.

Motivation & Context

The release process had three defects that only show up under failure.

Releases went public before they had artifacts. The release was created by hand in the GitHub UI, which created the tag, which triggered the workflow. For v1.12.1 the release was published at 07:36:38 and GoReleaser finished uploading at 07:44:55 — roughly eight minutes during which the release existed, was marked "Latest", and contained nothing. The Terraform Registry ingests on the release webhook, so it could see a release with no SHA256SUMS and no archives. The acceptance-test job was wired as a needs: dependency, which reads like a gate, but it ran after the release was already public: a test failure left a published, empty release for someone to notice and delete.

A failed release could not be re-run. With mode: append, every asset that already existed came back 422 Validation Failed [already_exists], and GoReleaser's retry loop just re-issued the same doomed request. Run 28953853165 shows all nine assets failing within three seconds. Recovery meant deleting assets by hand.

The toolchain was pinned to an end-of-life release. goreleaser-action@v5.1.0 resolves to GoReleaser v1.26.2, the last v1, and every release already logged DEPRECATED: changelog.skip. Separately, every action in the release path targeted Node 20, which GitHub is removing.

Changes Made

Release ordering and verification (.github/workflows/release.yaml, .goreleaser.yml)

  • Releases are now cut by pushing a tag; GoReleaser creates the release as a genuine draft, and a new provider-release_publish job publishes it only after verification. draft and replace_existing_draft previously had no effect, because GoReleaser was appending to a release a human had already created.
  • The verification step downloads every asset off the draft, runs sha256sum --check --strict over SHA256SUMS, and asserts the expected archive count before flipping the release to published. The count check is separate on purpose: the checksum file only covers what GoReleaser actually built, so it cannot catch a silently shrunken matrix.
  • New provider-release_preflight job rejects a run that is not on a tag, and refuses to re-release a tag whose release is already published — Registry versions are immutable, so replacing artifacts underneath one breaks every consumer that has it in a lock file. It runs ahead of the acceptance tests so both failures cost seconds rather than five minutes against the production API.
  • release.mode changed from append to replace, making re-runs idempotent while a release is still a draft.

Build matrix and version stamping (.goreleaser.yml, main.go, internal/acctest/acctest.go)

  • Builds 13 platforms instead of 6, adding linux/386, linux/arm, windows/386 and all four freebsd targets. An uncovered platform is a hard terraform init failure, not a degraded experience. darwin/386, darwin/arm and windows/arm are explicitly ignored — they no longer exist in modern Go, and HashiCorp's scaffolding template only excludes the first.
  • main.go declared const Version = "0.0.1" while the ldflags targeted main.version. The linker cannot write a const and silently ignores a missing symbol, so every release since has shipped reporting 0.0.1. It is now a var and takes the value from the tag. The dead -X main.commit flag is dropped, since nothing surfaces a commit and the symbol was being stripped.
  • acctest.go passes "test" as the version, which is what the struct comment in provider.go already claimed happened.

Toolchain (.github/workflows/community.yml, .github/workflows/test.yaml, Makefile)

  • Migrated to GoReleaser v2: version: 2, archives.format to formats, and changelog.skip to changelog.disable. These are coupled — v2 rejects the old config and v1 rejects the new one, so the config and the action bump have to land together.
  • Bumped every action in community.yml to a Node 24 release, keeping the existing SHA-pin-with-version-comment style. goreleaser-action was pinned by tag while its neighbours used SHAs; it is now consistent. test.yaml keeps its floating-major style: checkout@v7, setup-go@v7, setup-terraform@v4.
  • The release target in the Makefile used --rm-dist, --skip-publish and --skip-sign, all removed in v2. It now uses --clean --skip=publish,sign, so the local dry run matches what CI runs.

Release notes and documentation (CHANGELOG.md, README.md, CONTRIBUTING.md)

  • Release bodies were being generated by whoever created the release clicking GitHub's "Generate release notes" button. Now that GoReleaser creates the release, changelog.use: github-native produces the same body without a human step.
  • CHANGELOG.md last changed in March 2024, 53 releases ago, while README.md still pointed users at it for release notes. It is marked as a historical archive covering up to 0.1.0, and the README points at the releases page.
  • CONTRIBUTING.md gains the tag-push release flow, the re-run rules, the pre-release tag convention, and a note that EXPECTED_ZIPS tracks the build matrix. The instruction to update CHANGELOG.md before merging is replaced with guidance that PR titles become the release notes, which is now literally true.
  • Removed the dead .github/workflows/.old/release-old.yml.

Unrelated test fix, needed for CI to pass (internal/resource/workflow_from_template, internal/resource/workflow_git)

  • The API started rejecting terraform_version = "1.6.0" with 400: Terraform 1.6.0 is above the highest managed version (1.5.7). The same test literals passed on main in the scheduled run on 24 Aug (32689518464), so the managed version list changed server-side rather than anything here. The nine affected literals are pinned to 1.5.7, which keeps the update steps meaningful (they still move 1.5.0 to 1.5.7) and matches the version already used by the stack_template_revision and workflow_template_revision tests, both of which passed in the failing run. If the cap to 1.5.7 was not intentional, this pin hides a real regression and should be reverted in favour of an API fix.

Testing

  • goreleaser check passes on v2.18.0; v1.26.2 now rejects the config, confirming the config and action bump cannot drift apart
  • Full 13-platform snapshot build succeeds with zero deprecation warnings
  • Archive names are byte-identical between v1.26.2 and v2.18.0, so Registry ingestion is unaffected
  • Verification logic exercised against a reconstructed post-download asset set: passes on intact artifacts, rejects a corrupted archive, and rejects a missing platform
  • Version injection confirmed in a built binary — tag v9.9.9 produces -X main.version=9.9.9 and the string is present
  • actionlint clean on the release path; the two findings in test-api.yaml are pre-existing
  • go build, gofmt and make test show no regressions against unmodified main
  • test.yaml runs green on this PR with the bumped checkout@v7 / setup-go@v7 / setup-terraform@v4 actions
  • Cut a pre-release tag from this branch and confirm the full pipeline end to end
  • Confirm the published release body is non-empty, since the github-native path has never run

Risks & Edge Cases

  • The pipeline has never run end to end. Every part is verified locally, but tag push to draft to verify to publish has only executed in pieces. The first release cut this way is worth watching, especially the release body.
  • EXPECTED_ZIPS in release.yaml is manually coupled to the matrix in .goreleaser.yml. That is deliberate — it is what catches a silently shrunken build — but adding a platform without updating it will fail the release. Documented in CONTRIBUTING.md.
  • GoReleaser floats within v2. goreleaser-action@v7.2.3 defaults to ~> v2. Artifact naming has been stable and the new verification step would catch drift, but it can be pinned to ~> v2.18 if reproducibility matters more.
  • Pre-existing and unaddressed: the acceptance tests fail roughly 10% of the time on main, always with a 400 Bad Request returned as an nginx HTML page during resource creation. That is a gateway-level rejection rather than a provider or test bug, so no retry was added — masking it would remove the signal. It does mean roughly one release in ten will need a re-run, which is now a single click.

Deployment Notes

  • How releases are cut changes. Do not create the GitHub release by hand any more — that is what published an empty release before any artifact existed. Push a tag instead:
    git tag v1.13.0 && git push origin v1.13.0
    
    The workflow creates the draft, verifies it, and publishes it.
  • Re-running a failed release is now safe while it is still a draft; existing assets are replaced rather than rejected. Once a release is published it is final — ship a new patch version rather than re-running against it.
  • Pre-release tags should use a dot before the number (v1.13.0-beta.1, not -beta-1) so semantic versioning orders them numerically.
  • No new secrets. GPG_PRIVATE_KEY and the SG_PRD_* values are unchanged.

@hllvc hllvc self-assigned this Aug 31, 2026
hllvc added 2 commits August 31, 2026 17:48
Publish the GitHub release only after every artifact is downloaded back
and checked against SHA256SUMS, so the Registry never ingests a release
that has no assets attached yet. Switch goreleaser to mode: replace so
re-runs stop dying on 422 already_exists, and add a preflight that
refuses to re-release an already published tag.

Also build 13 platforms instead of 6, take the version from the tag
(main.version has to be a var for -ldflags to apply at all), migrate to
goreleaser v2, and move every action onto a node24 release.
Release notes are generated when a release is published now, so mark
CHANGELOG.md as a historical archive covering up to 0.1.0 and point the
README at the releases page instead of a file that stopped 53 releases
ago. CONTRIBUTING gets the tag-push release flow and the pre-release tag
convention.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant