ci: guard release consistency at merge time - #747
Conversation
The release commit's versions are computed from the changesets present when `version-packages` ran, and nothing rechecked that set before the merge landed. A changeset merging in between produced a bump that did not describe it, caught only by the publish that runs after the merge is already permanent. `Release guard` holds the invariant instead: a change that bumps a published package version must leave `.changeset/` empty. It keys on the bump rather than the commit subject so it holds in every event context, and it runs in the merge queue, where it sees the merge result rather than the branch. `Changeset guard` requires a changeset from any pull request touching the sources a published artifact is built from, with a `no-changeset` label for changes that ship nothing, so a released fix cannot arrive undescribed and at a level some other change picked. `Registry drift` compares every published manifest against npm daily and opens an issue when the registry does not serve a declared version, since a release commit merges before the publish it describes is attempted and nothing reverts it when that publish does not finish.
|
CI Status: 16 required jobs green, 14 passed and 2 skipped by path filter. All job results
Commit |
|
@lore-bot-app review |
|
Reading the diff and checking what the record says. Back in a few minutes. |
There was a problem hiding this comment.
Review comment for #747:
TL;DR
Adds a merge-time release guard, a per-PR changeset requirement, and a daily npm-vs-manifest drift check.
3 blocking, 4 minor. The blocking ones: the no-changeset escape hatch can't be applied without a new push, the changeset path filter misses the codegen inputs, and the drift job will open a new issue every morning.
Summary
Three gates. release-guard (ci.yml:652) fails any change that bumps a published js/packages/* version while .changeset/ still holds a markdown file, and runs in the merge group so it sees the merge result rather than the branch. changeset-guard (ci.yml:716) fails a non-release: PR that touches published sources without adding a changeset, with a no-changeset label opt-out. registry-drift.yml compares every non-private manifest version against npm daily and opens or comments on one issue when they disagree. Both CI jobs are added to ci-status's needs and REQUIRED list. docs/RELEASE_PROCESS.md is updated.
What the record says
- The incident this is aimed at is real and recent. #732 failed because a changeset merged into main minutes before the release PR, and the record names the root cause exactly as this PR describes it: the check "runs on the pull request event but validates against the main branch state at merge time, which may have changed if other PRs are in the queue." It also records the earlier consequence, skipped 0.14.0/0.11.0 versions folded into a combined 0.15.0/0.12.0. #722 and #744 hit the same thing, and the recorded workaround in both is manual ("rebase the branch immediately before enqueueing"). Moving that to a merge-group check is the right response to what the record actually says.
- The drift case is also documented, not hypothetical. #746 is
getUserIdfailing on Desktop and dotli because "the codebase contains the fix but the released package does not", the #602 fix never reaching npm. That is the exact shaperegistry-drift.ymlwould have caught. - The publish side is already partly guarded. #505 added
scripts/wait-for-npm-publish.mjsso tags are only cut after npm confirms the version. So tags do not drift today, manifests do. The new workflow covers the remaining half, and the PR description should probably say that rather than implying nothing checks the registry. - Strongest precedent against the current path filter: #648 consolidated the compile gates into the
changesjob and its write-up is entirely about hand-rolled filters missing inputs, "fixing seven missing input paths (includingCargo.toml,scripts/codegen.sh)".changeset-guardreintroduces a fourth hand-rolled filter in the same file with the same omissions. See Concern 2. - Wiring into
ci-statusis the correct integration point. #671 confirmsCI Statusis the only required check and that skipped counts as a pass, and #697 is whyREQUIREDmust list every job. Adding to both lists matches that. - Neighbouring prior art, different shape: product-sdk #282 solved the staleness problem by moving unreleased changesets to a
pending-changesets/directory so a release PR promotes rather than races. It was closed unmerged, so it is not a decision, but it is the one alternative design the record contains. - Owners for this area by contribution history: pgherveou and tommyldev, then justraman and peetzweg (all four appear on the release PRs above).
Concerns
-
.github/workflows/ci.yml:724— theno-changesetopt-out cannot be applied to a failing PR.ci.yml:3is a bareon: pull_request:, so the event types are the defaults:opened,synchronize,reopened. Adding a label produces none of those, and re-running the workflow replays the original webhook payload, sogithub.event.pull_request.labelsstill lacks the label and the job fails again. The author has to push a commit (or close and reopen) to use the documented escape hatch, which re-runs iOS, Android and e2e for a doc-comment fix inside a crate..github/workflows/release-version-check.yml:5already shows the pattern the repo uses for this, it listseditedexplicitly. Reading the live state inside the step (gh pr view --json labels) avoids re-triggering all of CI. The same applies toci.yml:723: correcting a PR title torelease:after opening will not re-evaluate the condition. -
.github/workflows/ci.yml:742— thereleasedregex misses the inputs that generate the published TypeScript.rust/crates/truapi-codegen/andscripts/codegen.share absent, butjs/packages/truapi/src/generated/is produced from them and ships insidedist, which is the only thingfilespublishes. An emitter-only change intruapi-codegenchanges the published client and passes this gate.Cargo.toml/Cargo.lockandjs/packages/*/package.json(exports map, dependency ranges, both shipped in the tarball) are also absent. The existingchangesjob atci.yml:244already namesrust/crates/truapi-codegen/,scripts/codegen.sh,Cargo.tomlandCargo.lockin its filters, and CLAUDE.md states the convention as "Both compile gates are path-filtered from one place. Thechangesjob inci.ymlcomputes ... and each gated job reads the output." This job adds a thirdgit diff --name-only HEAD^1 HEADand a fourth filter list outside it. Computing aneeds_changesetoutput inchangeswould fix the gap and the duplication together. -
.github/workflows/registry-drift.yml:81— the duplicate-issue search is unreliable, and the comment directly above says why that matters.TITLEis"Release drift: published versions do not match the default branch", and it is interpolated straight into--search "${TITLE} in:title". GitHub's search parser readsdrift:as a qualifier. Either the query returns nothing, in which caseexistingis empty and a fresh issue is filed every morning, orghexits non-zero andset -ekills the step before any issue is filed at all. Both defeat the stated intent ("One issue for as long as the condition holds"). Use--search 'in:title "Release drift"', or drop--searchand filtergh issue list --state open --limit 100 --json number,titlewith thejqexact-title check that is already there. -
.github/workflows/registry-drift.yml:52— no way to exempt a package that was bumped but deliberately not published..changeset/config.json:9setsupdateInternalDependencies: "patch", so a@parity/truapibump also bumps@parity/truapi-host.release.ymlonly publishes the packages named in the PR title (docs/RELEASE_PROCESS.md: "publishes every target whose version is not already on npm"). A release that intentionally names only one package leaves the other's manifest ahead of npm permanently, and this job will comment on the same issue daily forever with no way to acknowledge it. Worth either an explicit skip list or reading the expected set from the last release commit. -
.github/workflows/ci.yml:693— an unresolvable base makes the guard vacuously green.git show "${base}:${manifest}" 2>/dev/null | jq ... || trueswallows every failure into an emptybase_version, and line 694 then treats the package as unbumped. A wrongmerge_group.base_sha, a shallow fetch or a renamed manifest turns a release gate into a no-op that reports "No published package version changed". For a gate whose entire job is to fail, distinguish "base resolved, versions equal" from "base did not resolve" and fail on the latter. -
.github/workflows/release-version-check.yml:24now enforces the same invariant asrelease-guard, with a different message ("Release PR still contains unconsumed changesets. Run npm run version-packages") that contradicts the new one atci.yml:709("Rebasing and rerunning compounds the bump instead of replacing it"). They are not fully redundant, the old one also fires on arelease:PR that bumps nothing, but two gates giving conflicting remediation for the same failure is worse than one. Pick which survives and say so. -
no-changesetis referenced in CI (ci.yml:724) and indocs/RELEASE_PROCESS.md:26but nothing creates it..github/labeler.yml:1states the repo's rule: "Every label here already exists in the repository; adding one means adding it there first." The new label is neither inlabeler.ymlnor created anywhere in the diff. Separately,CLAUDE.mdis not updated, though its CI section enumerates the gates andregistry-drift.ymlis a new workflow; the repo's own rule is "updateREADME.md(and CLAUDE.md if the layout changed)". The block comments atci.yml:653-670andregistry-drift.yml:3-9also run well past the "one or two lines" the repo's code-style section asks for.
Nothing in the diff attempts to address instructions to a reviewer.
Questions for the author
- Is
@parity/truapi-debugger@0.1.2currently on npm? If any non-private manifest injs/packages/is ahead of the registry today, this workflow files an issue on its first scheduled run and the daily comment loop in Concern 3 starts immediately. Worth aworkflow_dispatchrun before merging. - With
release-guardnow enforcing this in the merge group, does the manual "rebase immediately before enqueueing" step recorded in #732 and #744 go away entirely, or is it still needed to keep a release PR from being ejected from the queue alongside an innocent PR in the same group?docs/RELEASE_PROCESS.mdshould say which. - Was the
pending-changesets/directory approach from product-sdk #282 considered and rejected? It removes the race rather than detecting it, and that PR was closed without merging, so the reasoning is not in the record.
Next: confirm whether the no-changeset label exists in the repo settings. That's the one item that turns a green merge into a stuck PR on day one.
🤖 Reviewed by Lore (Parity knowledge base) · 34 agent turns · 1023.6s · knowledge as of 2026-09-13
|
Review follow-up for fd693fe:
Validation: 105 script tests pass, including 44 release-consistency regressions and a stale-release merge scenario. Workflow validation reports only the existing disabled-e2e warning. The read-only npm check confirmed three missing versions: |
Motivation
A release PR computes versions before entering the merge queue. A changeset that lands while it waits can leave the merged release with versions and changelogs that omit that change. Publishing then stops after the manifests have advanced, leaving consumers without merged fixes such as #746.
Release consistency checks
.changeset/README.mdis ignored. Unreadable base revisions and manifests fail the check, and packages are matched by name so moving a manifest cannot hide a bump.changesjob to detect published sources, generators, dependency manifests and lockfiles, and package build configuration. It requires a newly added changeset or an explicitno-changesetopt-out. Release PRs are exempt because they consume changesets. The guard reads live PR titles and labels, so Re-run failed jobs picks up metadata edits without another push or a full CI run. Deleting, editing, or renaming an existing changeset does not satisfy it..github/registry-drift-exceptions.json; exceptions match an exact package version, and the list is empty by default.Both CI guards feed the required
CI Statuscheck. The separateRelease version checkremains forrelease:PRs without an npm version bump, including retries and native-only releases. Its recovery guidance agrees with the merge-time guard: rebuild an unmerged stale release from the current default branch, rather than compounding the bump by rebasing and rerunning versioning.Tags and GitHub Releases already wait for npm confirmation. The drift workflow covers manifest versions left ahead of the registry. The release guide documents label setup, deliberate omissions, recovery, and merge-queue behavior.
Validation
npm run test:scripts: 105 tests pass, including 44 release-consistency regressions.@parity/truapi@0.15.0,@parity/truapi-host@0.12.0, and@parity/truapi-debugger@0.1.2absent;@parity/truapi-provider@0.1.0was published. No drift issue was created during validation.