Skip to content

fix(ci): review pull requests from forks - #2537

Merged
happyhuman merged 1 commit into
mainfrom
ci/review-fork-prs
Aug 17, 2026
Merged

fix(ci): review pull requests from forks#2537
happyhuman merged 1 commit into
mainfrom
ci/review-fork-prs

Conversation

@happyhuman

Copy link
Copy Markdown
Collaborator

Problem

Of the last 60 PRs, excluding Dependabot and maintainers, 10 of 11 came from forks — and every one was skipped by the AI reviewers. The gate required head.repo.fork == false, so automated review has effectively only ever covered maintainer branches. #2530 and #2507 are current examples.

Widening the author allowlist does not fix it. A fork pull_request gets no secrets, a read-only GITHUB_TOKEN and no OIDC token, so Workload Identity auth and comment posting cannot work on that event — it would just fail later instead of skipping early.

Change

pull_requestpull_request_target opened, reopened, synchronize, so fork PRs are reviewed on open and every push
Fork check + author allowlist Removed from the automatic path; kept on the comment re-invoke path
New-file-only skip Removed
pull_request_review / _review_comment triggers Removed
4 conditional checkouts Collapsed into 1 unconditional base-repo checkout
new_files_only_guard Renamed diff_size_guard
@gemini-cli /review Renamed @ai-review

Net 128 insertions / 249 deletions.

Why drop the new-file-only skip: it fired whenever every file was added, which covers 4 of those 10 fork PRs. A first-time contributor submitting a new recipe is exactly who benefits most from automated feedback. Without this, changing the trigger alone would have fixed 6 of 10.

Why the duplicate check rows disappear: the two review triggers fired on every review event regardless of body, so each review spawned three runs that immediately skipped, each with its own check row — the (pull_request_review) duplicates on #2530. issue_comment re-invocation is unaffected.

Security

pull_request_target is safe here because nothing in these workflows executes the PR's code. The diff and metadata arrive through gh as data, the agent runs in an empty scratch directory, and the checkout is the base repo — already true since #2536, now unconditional so the invariant can't be broken by adding an event. Carried the dependabot-auto-merge.yml convention: # zizmor: ignore[dangerous-triggers] on the on: line, with reassess if a step is ever added that runs anything out of the checkout.

Residual risk is prompt injection via the diff. Bounded: findings are validated against the diff (path + line must be an added line) before posting, so the worst case is a misleading comment — not code execution or token access.

pull_request_target also bypasses fork-approval, so a PR opening now triggers 3 model calls with no maintainer click.

Verification

Check Result
Caller/core input contract, all 3 callers OK — no undeclared or missing inputs
Dangling steps.* refs none (7 ids, all resolve)
Per-event step paths pull_request_target / issue_comment / workflow_dispatch each get 13 steps and exactly one PR-details step
#2530 (fork, 33 all-added) skip=falsewould be reviewed
#2507 (fork, 9 all-added) skip=falsewould be reviewed
#2535 (804 files) skip=true — size guard intact
300 / 301 file boundary pass / skip
gh api failure exit 1, no output — fails loud, never silently skips
@ai-review instruction stripping @ai-review focus on Xfocus on X
uv run --frozen pytest -q 1120 passed

Guard behaviour was checked by extracting the step's run: block and executing it against the live PRs with gh pr comment stubbed, so nothing was posted.

⚠️ Cannot be tested on this PR

pull_request_target runs the workflow from the default branch, so this PR still exercises the old logic. First real test is after merge — suggest re-running #2530 to confirm, with revert-on-main as rollback.

Follow-up

ai-issue-automated-triage.yml still uses @gemini-cli /triage, now the only @gemini-cli left. Left out to keep this PR to PR review; happy to rename it to @ai-triage separately.

Of the last 60 pull requests, excluding Dependabot and maintainers, 10 of 11
came from forks — and every one of them was skipped by the AI reviewers. The
gate required `head.repo.fork == false`, so in practice automated review
covered maintainer branches and nothing else.

Widening the author allowlist would not have helped: a fork `pull_request`
gets no secrets, a read-only token and no OIDC token, so Workload Identity
auth and comment posting cannot work on that event at all. `pull_request_target`
is the only trigger that can, and it is safe here for a specific reason —
nothing in these workflows executes the pull request's code. The diff and
metadata arrive through `gh` as data, the agent runs in an empty scratch
directory, and the checkout is the base repo. That last property was already
true (#2536); this change makes it unconditional so the invariant cannot be
broken by adding an event. The remaining exposure is prompt injection via the
diff, which is bounded: findings are validated against the diff before posting,
so the worst case is a misleading comment, not code execution or token access.

Also drops the new-file-only skip. It fired on any pull request where every
file was `added`, which is 4 of those 10 fork pull requests — a first-time
contributor submitting a new recipe is exactly who benefits most from
automated feedback, and "a maintainer reviews new recipes anyway" is weakest
precisely there. Without this, switching the trigger would have fixed 6 of 10.

Two clean-ups fall out of the above. The `pull_request_review` and
`pull_request_review_comment` triggers are gone: they fired on every review
event regardless of body, so each review submitted on a pull request spawned
three runs that immediately skipped, and each produced its own check row —
the duplicate `(pull_request_review)` entries visible on #2530. Manual
re-invocation keeps working through `issue_comment`, which does not have that
problem. And the guard no longer has anything to say about new files, so it is
renamed for what it now does: refuse a diff GitHub will not serve.

The re-invoke phrase becomes `@ai-review`. It never referred to a bot — it is
a literal substring match — but it has named the wrong tool since the
Antigravity migration in #2509.

Note this cannot be exercised on its own pull request: `pull_request_target`
runs the workflow from the default branch, so the new behaviour only takes
effect once merged. Re-run an existing fork PR (#2530 is a good candidate) to
confirm, and revert on main if it misbehaves.
@happyhuman
happyhuman merged commit 10e5333 into main Aug 17, 2026
10 checks passed
@happyhuman
happyhuman deleted the ci/review-fork-prs branch August 17, 2026 22:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant