fix(ci): review pull requests from forks - #2537
Merged
Merged
Conversation
Of the last 60 pull requests, excluding Dependabot and maintainers, 10 of 11 came from forks — and every one of them was skipped by the AI reviewers. The gate required `head.repo.fork == false`, so in practice automated review covered maintainer branches and nothing else. Widening the author allowlist would not have helped: a fork `pull_request` gets no secrets, a read-only token and no OIDC token, so Workload Identity auth and comment posting cannot work on that event at all. `pull_request_target` is the only trigger that can, and it is safe here for a specific reason — nothing in these workflows executes the pull request's code. The diff and metadata arrive through `gh` as data, the agent runs in an empty scratch directory, and the checkout is the base repo. That last property was already true (#2536); this change makes it unconditional so the invariant cannot be broken by adding an event. The remaining exposure is prompt injection via the diff, which is bounded: findings are validated against the diff before posting, so the worst case is a misleading comment, not code execution or token access. Also drops the new-file-only skip. It fired on any pull request where every file was `added`, which is 4 of those 10 fork pull requests — a first-time contributor submitting a new recipe is exactly who benefits most from automated feedback, and "a maintainer reviews new recipes anyway" is weakest precisely there. Without this, switching the trigger would have fixed 6 of 10. Two clean-ups fall out of the above. The `pull_request_review` and `pull_request_review_comment` triggers are gone: they fired on every review event regardless of body, so each review submitted on a pull request spawned three runs that immediately skipped, and each produced its own check row — the duplicate `(pull_request_review)` entries visible on #2530. Manual re-invocation keeps working through `issue_comment`, which does not have that problem. And the guard no longer has anything to say about new files, so it is renamed for what it now does: refuse a diff GitHub will not serve. The re-invoke phrase becomes `@ai-review`. It never referred to a bot — it is a literal substring match — but it has named the wrong tool since the Antigravity migration in #2509. Note this cannot be exercised on its own pull request: `pull_request_target` runs the workflow from the default branch, so the new behaviour only takes effect once merged. Re-run an existing fork PR (#2530 is a good candidate) to confirm, and revert on main if it misbehaves.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Of the last 60 PRs, excluding Dependabot and maintainers, 10 of 11 came from forks — and every one was skipped by the AI reviewers. The gate required
head.repo.fork == false, so automated review has effectively only ever covered maintainer branches. #2530 and #2507 are current examples.Widening the author allowlist does not fix it. A fork
pull_requestgets no secrets, a read-onlyGITHUB_TOKENand no OIDC token, so Workload Identity auth and comment posting cannot work on that event — it would just fail later instead of skipping early.Change
pull_request→pull_request_targetopened, reopened, synchronize, so fork PRs are reviewed on open and every pushpull_request_review/_review_commenttriggersnew_files_only_guarddiff_size_guard@gemini-cli /review@ai-reviewNet 128 insertions / 249 deletions.
Why drop the new-file-only skip: it fired whenever every file was
added, which covers 4 of those 10 fork PRs. A first-time contributor submitting a new recipe is exactly who benefits most from automated feedback. Without this, changing the trigger alone would have fixed 6 of 10.Why the duplicate check rows disappear: the two review triggers fired on every review event regardless of body, so each review spawned three runs that immediately skipped, each with its own check row — the
(pull_request_review)duplicates on #2530.issue_commentre-invocation is unaffected.Security
pull_request_targetis safe here because nothing in these workflows executes the PR's code. The diff and metadata arrive throughghas data, the agent runs in an empty scratch directory, and the checkout is the base repo — already true since #2536, now unconditional so the invariant can't be broken by adding an event. Carried thedependabot-auto-merge.ymlconvention:# zizmor: ignore[dangerous-triggers]on theon:line, with reassess if a step is ever added that runs anything out of the checkout.Residual risk is prompt injection via the diff. Bounded: findings are validated against the diff (path + line must be an added line) before posting, so the worst case is a misleading comment — not code execution or token access.
pull_request_targetalso bypasses fork-approval, so a PR opening now triggers 3 model calls with no maintainer click.Verification
steps.*refspull_request_target/issue_comment/workflow_dispatcheach get 13 steps and exactly one PR-details stepskip=false→ would be reviewedskip=false→ would be reviewedskip=true— size guard intactgh apifailure@ai-reviewinstruction stripping@ai-review focus on X→focus on Xuv run --frozen pytest -qGuard behaviour was checked by extracting the step's
run:block and executing it against the live PRs withgh pr commentstubbed, so nothing was posted.pull_request_targetruns the workflow from the default branch, so this PR still exercises the old logic. First real test is after merge — suggest re-running #2530 to confirm, with revert-on-main as rollback.Follow-up
ai-issue-automated-triage.ymlstill uses@gemini-cli /triage, now the only@gemini-clileft. Left out to keep this PR to PR review; happy to rename it to@ai-triageseparately.