Skip to content

ci: guard release consistency at merge time - #747

Merged
pgherveou merged 2 commits into
mainfrom
ci/release-consistency-guards
Sep 14, 2026
Merged

pgherveou merged 2 commits into
mainfrom
ci/release-consistency-guards

Conversation

@TorstenStueber

@TorstenStueber TorstenStueber commented Sep 13, 2026

Copy link
Copy Markdown
Collaborator

Motivation

A release PR computes versions before entering the merge queue. A changeset that lands while it waits can leave the merged release with versions and changelogs that omit that change. Publishing then stops after the manifests have advanced, leaving consumers without merged fixes such as #746.

Release consistency checks

  • Release guard checks published npm package versions against the merge base on pull requests and merge groups, and against the pre-push revision on pushes. A version change must consume every pending changeset; .changeset/README.md is ignored. Unreadable base revisions and manifests fail the check, and packages are matched by name so moving a manifest cannot hide a bump.
  • Changeset guard uses the shared changes job to detect published sources, generators, dependency manifests and lockfiles, and package build configuration. It requires a newly added changeset or an explicit no-changeset opt-out. Release PRs are exempt because they consume changesets. The guard reads live PR titles and labels, so Re-run failed jobs picks up metadata edits without another push or a full CI run. Deleting, editing, or renaming an existing changeset does not satisfy it.
  • Registry drift checks default-branch manifests against npm daily and on manual dispatch. Missing versions produce one open issue, updated only when the report changes. Registry lookup errors fail the job without modifying an issue. Intentionally unpublished versions can be documented with a reason in .github/registry-drift-exceptions.json; exceptions match an exact package version, and the list is empty by default.

Both CI guards feed the required CI Status check. The separate Release version check remains for release: PRs without an npm version bump, including retries and native-only releases. Its recovery guidance agrees with the merge-time guard: rebuild an unmerged stale release from the current default branch, rather than compounding the bump by rebasing and rerunning versioning.

Tags and GitHub Releases already wait for npm confirmation. The drift workflow covers manifest versions left ahead of the registry. The release guide documents label setup, deliberate omissions, recovery, and merge-queue behavior.

Validation

  • npm run test:scripts: 105 tests pass, including 44 release-consistency regressions.
  • The merge-history regression passes on the release branch and rejects the merge result after a late changeset lands.
  • Workflow linting reports only the existing constant-false condition on the deliberately disabled e2e job. Shell syntax and formatting checks for the changed workflows pass.
  • A local, read-only npm comparison found @parity/truapi@0.15.0, @parity/truapi-host@0.12.0, and @parity/truapi-debugger@0.1.2 absent; @parity/truapi-provider@0.1.0 was published. No drift issue was created during validation.

The release commit's versions are computed from the changesets present when
`version-packages` ran, and nothing rechecked that set before the merge landed.
A changeset merging in between produced a bump that did not describe it, caught
only by the publish that runs after the merge is already permanent.

`Release guard` holds the invariant instead: a change that bumps a published
package version must leave `.changeset/` empty. It keys on the bump rather than
the commit subject so it holds in every event context, and it runs in the merge
queue, where it sees the merge result rather than the branch.

`Changeset guard` requires a changeset from any pull request touching the
sources a published artifact is built from, with a `no-changeset` label for
changes that ship nothing, so a released fix cannot arrive undescribed and at a
level some other change picked.

`Registry drift` compares every published manifest against npm daily and opens
an issue when the registry does not serve a declared version, since a release
commit merges before the publish it describes is attempted and nothing reverts
it when that publish does not finish.
@TorstenStueber
TorstenStueber requested a review from a team September 13, 2026 23:47
@github-actions github-actions Bot added documentation Improvements or additions to documentation github_actions Pull requests that update GitHub Actions code labels Sep 13, 2026
@github-actions

github-actions Bot commented Sep 14, 2026

Copy link
Copy Markdown

CI Status: 16 required jobs green, 14 passed and 2 skipped by path filter.

All job results
job result
android-bindings success
changes success
changeset-guard skipped
codegen success
e2e skipped
explorer success
ios-bindings success
ios-swift success
licenses success
playground success
release-guard success
rust success
ts-client success
ts-debugger success
ts-host success
wasm-provider success

Commit fd693fe3 · run log

@TarikGul

Copy link
Copy Markdown
Member

@lore-bot-app review

@lore-bot-app

lore-bot-app Bot commented Sep 14, 2026

Copy link
Copy Markdown

Reading the diff and checking what the record says. Back in a few minutes.

@lore-bot-app lore-bot-app Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review comment for #747:


TL;DR
Adds a merge-time release guard, a per-PR changeset requirement, and a daily npm-vs-manifest drift check.
3 blocking, 4 minor. The blocking ones: the no-changeset escape hatch can't be applied without a new push, the changeset path filter misses the codegen inputs, and the drift job will open a new issue every morning.

Summary
Three gates. release-guard (ci.yml:652) fails any change that bumps a published js/packages/* version while .changeset/ still holds a markdown file, and runs in the merge group so it sees the merge result rather than the branch. changeset-guard (ci.yml:716) fails a non-release: PR that touches published sources without adding a changeset, with a no-changeset label opt-out. registry-drift.yml compares every non-private manifest version against npm daily and opens or comments on one issue when they disagree. Both CI jobs are added to ci-status's needs and REQUIRED list. docs/RELEASE_PROCESS.md is updated.

What the record says

  • The incident this is aimed at is real and recent. #732 failed because a changeset merged into main minutes before the release PR, and the record names the root cause exactly as this PR describes it: the check "runs on the pull request event but validates against the main branch state at merge time, which may have changed if other PRs are in the queue." It also records the earlier consequence, skipped 0.14.0/0.11.0 versions folded into a combined 0.15.0/0.12.0. #722 and #744 hit the same thing, and the recorded workaround in both is manual ("rebase the branch immediately before enqueueing"). Moving that to a merge-group check is the right response to what the record actually says.
  • The drift case is also documented, not hypothetical. #746 is getUserId failing on Desktop and dotli because "the codebase contains the fix but the released package does not", the #602 fix never reaching npm. That is the exact shape registry-drift.yml would have caught.
  • The publish side is already partly guarded. #505 added scripts/wait-for-npm-publish.mjs so tags are only cut after npm confirms the version. So tags do not drift today, manifests do. The new workflow covers the remaining half, and the PR description should probably say that rather than implying nothing checks the registry.
  • Strongest precedent against the current path filter: #648 consolidated the compile gates into the changes job and its write-up is entirely about hand-rolled filters missing inputs, "fixing seven missing input paths (including Cargo.toml, scripts/codegen.sh)". changeset-guard reintroduces a fourth hand-rolled filter in the same file with the same omissions. See Concern 2.
  • Wiring into ci-status is the correct integration point. #671 confirms CI Status is the only required check and that skipped counts as a pass, and #697 is why REQUIRED must list every job. Adding to both lists matches that.
  • Neighbouring prior art, different shape: product-sdk #282 solved the staleness problem by moving unreleased changesets to a pending-changesets/ directory so a release PR promotes rather than races. It was closed unmerged, so it is not a decision, but it is the one alternative design the record contains.
  • Owners for this area by contribution history: pgherveou and tommyldev, then justraman and peetzweg (all four appear on the release PRs above).

Concerns

  1. .github/workflows/ci.yml:724 — the no-changeset opt-out cannot be applied to a failing PR. ci.yml:3 is a bare on: pull_request:, so the event types are the defaults: opened, synchronize, reopened. Adding a label produces none of those, and re-running the workflow replays the original webhook payload, so github.event.pull_request.labels still lacks the label and the job fails again. The author has to push a commit (or close and reopen) to use the documented escape hatch, which re-runs iOS, Android and e2e for a doc-comment fix inside a crate. .github/workflows/release-version-check.yml:5 already shows the pattern the repo uses for this, it lists edited explicitly. Reading the live state inside the step (gh pr view --json labels) avoids re-triggering all of CI. The same applies to ci.yml:723: correcting a PR title to release: after opening will not re-evaluate the condition.

  2. .github/workflows/ci.yml:742 — the released regex misses the inputs that generate the published TypeScript. rust/crates/truapi-codegen/ and scripts/codegen.sh are absent, but js/packages/truapi/src/generated/ is produced from them and ships inside dist, which is the only thing files publishes. An emitter-only change in truapi-codegen changes the published client and passes this gate. Cargo.toml / Cargo.lock and js/packages/*/package.json (exports map, dependency ranges, both shipped in the tarball) are also absent. The existing changes job at ci.yml:244 already names rust/crates/truapi-codegen/, scripts/codegen.sh, Cargo.toml and Cargo.lock in its filters, and CLAUDE.md states the convention as "Both compile gates are path-filtered from one place. The changes job in ci.yml computes ... and each gated job reads the output." This job adds a third git diff --name-only HEAD^1 HEAD and a fourth filter list outside it. Computing a needs_changeset output in changes would fix the gap and the duplication together.

  3. .github/workflows/registry-drift.yml:81 — the duplicate-issue search is unreliable, and the comment directly above says why that matters. TITLE is "Release drift: published versions do not match the default branch", and it is interpolated straight into --search "${TITLE} in:title". GitHub's search parser reads drift: as a qualifier. Either the query returns nothing, in which case existing is empty and a fresh issue is filed every morning, or gh exits non-zero and set -e kills the step before any issue is filed at all. Both defeat the stated intent ("One issue for as long as the condition holds"). Use --search 'in:title "Release drift"', or drop --search and filter gh issue list --state open --limit 100 --json number,title with the jq exact-title check that is already there.

  4. .github/workflows/registry-drift.yml:52 — no way to exempt a package that was bumped but deliberately not published. .changeset/config.json:9 sets updateInternalDependencies: "patch", so a @parity/truapi bump also bumps @parity/truapi-host. release.yml only publishes the packages named in the PR title (docs/RELEASE_PROCESS.md: "publishes every target whose version is not already on npm"). A release that intentionally names only one package leaves the other's manifest ahead of npm permanently, and this job will comment on the same issue daily forever with no way to acknowledge it. Worth either an explicit skip list or reading the expected set from the last release commit.

  5. .github/workflows/ci.yml:693 — an unresolvable base makes the guard vacuously green. git show "${base}:${manifest}" 2>/dev/null | jq ... || true swallows every failure into an empty base_version, and line 694 then treats the package as unbumped. A wrong merge_group.base_sha, a shallow fetch or a renamed manifest turns a release gate into a no-op that reports "No published package version changed". For a gate whose entire job is to fail, distinguish "base resolved, versions equal" from "base did not resolve" and fail on the latter.

  6. .github/workflows/release-version-check.yml:24 now enforces the same invariant as release-guard, with a different message ("Release PR still contains unconsumed changesets. Run npm run version-packages") that contradicts the new one at ci.yml:709 ("Rebasing and rerunning compounds the bump instead of replacing it"). They are not fully redundant, the old one also fires on a release: PR that bumps nothing, but two gates giving conflicting remediation for the same failure is worse than one. Pick which survives and say so.

  7. no-changeset is referenced in CI (ci.yml:724) and in docs/RELEASE_PROCESS.md:26 but nothing creates it. .github/labeler.yml:1 states the repo's rule: "Every label here already exists in the repository; adding one means adding it there first." The new label is neither in labeler.yml nor created anywhere in the diff. Separately, CLAUDE.md is not updated, though its CI section enumerates the gates and registry-drift.yml is a new workflow; the repo's own rule is "update README.md (and CLAUDE.md if the layout changed)". The block comments at ci.yml:653-670 and registry-drift.yml:3-9 also run well past the "one or two lines" the repo's code-style section asks for.

Nothing in the diff attempts to address instructions to a reviewer.

Questions for the author

  • Is @parity/truapi-debugger@0.1.2 currently on npm? If any non-private manifest in js/packages/ is ahead of the registry today, this workflow files an issue on its first scheduled run and the daily comment loop in Concern 3 starts immediately. Worth a workflow_dispatch run before merging.
  • With release-guard now enforcing this in the merge group, does the manual "rebase immediately before enqueueing" step recorded in #732 and #744 go away entirely, or is it still needed to keep a release PR from being ejected from the queue alongside an innocent PR in the same group? docs/RELEASE_PROCESS.md should say which.
  • Was the pending-changesets/ directory approach from product-sdk #282 considered and rejected? It removes the race rather than detecting it, and that PR was closed without merging, so the reasoning is not in the record.

Next: confirm whether the no-changeset label exists in the repo settings. That's the one item that turns a green merge into a stuck PR on day one.


🤖 Reviewed by Lore (Parity knowledge base) · 34 agent turns · 1023.6s · knowledge as of 2026-09-13

Comment thread .github/workflows/ci.yml Outdated
Comment thread .github/workflows/ci.yml Outdated
Comment thread .github/workflows/registry-drift.yml Outdated
Comment thread .github/workflows/registry-drift.yml Outdated
Comment thread .github/workflows/ci.yml Outdated
Comment thread .github/workflows/ci.yml
@pgherveou

Copy link
Copy Markdown
Collaborator

Review follow-up for fd693fe:

  • The changeset guard reads live PR metadata and uses the shared change filter, covering generators, dependency pins, and package build configuration. Deleted, edited, or renamed changesets do not satisfy it.
  • The release guard fails on unreadable base revisions or manifests and detects version changes across manifest moves. Both release checks remain, with consistent recovery guidance for unmerged release branches.
  • Drift reporting uses a paginated issue listing and leaves identical reports alone. Registry outages fail separately from missing versions. Intentional omissions require an explicit package-version exception with a reason; the exception list is empty.
  • The no-changeset repository label is available. After applying it to a failing PR, use Re-run failed jobs so the guard reads the current labels.

Validation: 105 script tests pass, including 44 release-consistency regressions and a stale-release merge scenario. Workflow validation reports only the existing disabled-e2e warning.

The read-only npm check confirmed three missing versions: @parity/truapi@0.15.0, @parity/truapi-host@0.12.0, and @parity/truapi-debugger@0.1.2. No drift issue was created and no version was exempted. Tags and GitHub Releases already wait for npm confirmation; this workflow detects manifests ahead of npm.

@pgherveou
pgherveou enabled auto-merge September 14, 2026 07:24
@pgherveou
pgherveou added this pull request to the merge queue Sep 14, 2026
Merged via the queue into main with commit cf1b28b Sep 14, 2026
23 checks passed
@pgherveou
pgherveou deleted the ci/release-consistency-guards branch September 14, 2026 07:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation github_actions Pull requests that update GitHub Actions code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants