feat(pr-review): let the reviewer report omitted companion changes - #10
Merged
Conversation
A change can be correct on every line it touches and still ship broken because a file it depends on was never updated. The reviewer could not report that class of defect at all, for three compounding reasons: - The compiler rejected any finding whose path was outside the PR's changed files, raising FINDING_ANCHOR_INVALID. That is not a demotion to body-only — it aborts the whole compile step, so a correct omission finding would have destroyed the review that carried it. - The prompt told the model to source anchors from the full PR diff and said nothing about reading past it, so a full review behaved like a diff review. - Neither the rubric nor the pre-mortem dimensions named the failure mode, and the pre-mortem's evidence verifier explicitly required every anchor to belong to the current diff — which refutes every omission finding by construction. Now the compiler accepts a path that exists in the checkout even when the PR did not touch it, and routes it into the review body, where findings without an inline anchor already go. The existing anti-hallucination guard is preserved as a file-existence check, with resolved paths confined to the workspace so a `../` or absolute path cannot read out of it. Rubric lens 4 asks what a complete version of the change touches and how to check it against repository history; the pre-mortem gains the stale-companion incident shape; and the contract states that the changed files bound which changes may be raised, not what may be read. The strong tier's turn budget goes 36 -> 44 to pay for that extra reading. Also adds a `debug_logs` input, default off. Both analysis steps hardcoded `show_full_output: false`, which makes "why did the review miss this?" unanswerable after the fact — the job log shows no tool calls, so there is no way to tell which files the reviewer opened and which it never read. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
A real miss prompted this. megaeth-labs/dist-docs#52 is a release-notes PR that announces a hardfork, tells operators to update their genesis file, and never adds
rex6Timetomainnet/genesis.jsonortestnet/genesis.json. Every prior hardfork release in that repo updated genesis in the same commit. The reviewer ran a full review with the pre-mortem enabled and returned ✅ Review clean.It was not a judgment error. The reviewer was structurally incapable of reporting it:
compile_reviewraisedFINDING_ANCHOR_INVALIDfor any finding whosepathwas not infull_pr_paths. That is not a demotion to body-only —PipelineErroraborts the compile step, so a correct omission finding would have destroyed the entire review that carried it.What changed
review_pipeline.py— a finding may name a file the PR did not touch, provided that file exists in the checkout. It has no RIGHT side in the diff, sofinding["inline"]is alreadyFalseand the existing renderer puts it in the review body; no new output path. The anti-hallucination guard that rejecting unchanged paths used to provide is preserved as a file-existence check, andtracked_file_existsconfines resolved paths to the workspace so a model-supplied../or absolute path cannot read outside it.compile_reviewtakes an injectablerepo_rootfor testability.rubric.md— new lens 4, Completeness — what the diff leaves undone: derive the companion set from the stated intent rather than the diff, the recurring shapes (a value declared in one file and consumed in another, docs promising behavior the data lacks, an enum arm added at one site but not its siblings, a version that must agree across files), andgit logon comparable earlier commits as the cheapest way to learn which files move together in this repo. Closes with the point that an unanchorable finding is not a weaker finding.premortem.md— dimension 1 gains the stale-companion incident shape, where nothing in the diff itself looks wrong; the Evidence Verifier gets the matching exception to its anchor rule.action.yml— contract item 1: the frozen scope bounds which changes may be raised, not what may be read. Item 7: how to anchor an omission, and not to downgrade it to an open question once verified. Strong-tier turn budget 36 → 44 to pay for the extra reading.review-output.schema.json—pathandlinedescriptions say how to anchor an omission.debug_logsinput (default off) — both analysis steps hardcodedshow_full_output: false. Diagnosing this miss meant inferring from a job log that shows no tool calls at all; there was no way to tell which files the reviewer opened and which it never read. Turning this on is the difference between answering that question in a minute and guessing.Testing
python3 -m unittest discover— 83 pass. New coverage: an omission finding on an unchanged-but-present file compiles to a body-only finding with verdictfindings; a path absent from the checkout still fails loudly; a../path pointing at a real file outside the workspace is rejected.Editing these files changes
pipeline_version/rubric_version, so open PRs across the org get one full re-review on the next event rather than an incremental one. That is the intended behavior here — the new lens should apply to work already in flight.🤖 Generated with Claude Code