Ignore Dependabot security updates in the failure watcher - #464
Conversation
📝 WalkthroughWalkthroughThe Dependabot failure watcher documents run categories and filters recent workflow runs to exclude security-driven and Node e2e failures while reporting version-update failures specifically. ChangesDependabot failure watcher
Estimated code review effort: 2 (Simple) | ~10 minutes Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Pull request overview
Updates the Dependabot Failure Watcher workflow to avoid being perpetually red due to Dependabot security update runs that frequently fail for reasons a repository PR can’t address, while keeping signal for version update failures.
Changes:
- Filters out Dependabot security-update workflow runs using
displayTitlepatterns (including a targeted denylist for/e2e/jsand/e2e/tsruns). - Bounds the GitHub CLI query server-side with
gh run list --createdand increases the limit to reduce the chance of missing in-window failures. - Expands inline documentation explaining Dependabot run types and why specific title substrings are “load-bearing”.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/dependabot-failure-watcher.yml:
- Around line 93-98: Update the run query in the workflow’s gh run list block to
detect truncation by requesting one sentinel row beyond the intended 500
results, then fail closed when that sentinel is present before filtering or
reporting success. Preserve the existing in-window query and ensure the sentinel
cannot be mistaken for a real run result.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 9cd77dfc-bd3f-4b00-afb5-d13b828ea222
📒 Files selected for processing (1)
.github/workflows/dependabot-failure-watcher.yml
| runs=$(gh run list \ | ||
| --repo "$REPO" \ | ||
| --workflow "Dependabot Updates" \ | ||
| --limit 100 \ | ||
| --json conclusion,createdAt,displayTitle,url \ | ||
| --jq "[.[] | select((.conclusion == \"failure\" or .conclusion == \"startup_failure\" or .conclusion == \"timed_out\") and .createdAt >= \"$since\")]") | ||
| --created ">=$since" \ | ||
| --limit 500 \ | ||
| --json conclusion,createdAt,displayTitle,url) |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== workflow file excerpt =="
sed -n '1,140p' .github/workflows/dependabot-failure-watcher.yml 2>/dev/null || true
echo
echo "== gh cli availability and version =="
if command -v gh >/dev/null 2>&1; then
gh --version | head -n 3
else
echo "gh not found"
fi
echo
echo "== search for dependabot failure watcher references =="
rg -n "dependabot-failure-watcher|Dependabot Updates|gh run list|--limit 500|created \)>=|since" .github workflow 2>/dev/null || rg -n "dependabot-failure-watcher|Dependabot Updates|gh run list|--limit 500|created \)>=|since" . 2>/dev/null || trueRepository: maxmind/libmaxminddb
Length of output: 7630
🌐 Web query:
GitHub CLI gh run list --limit default created date Actions API response per_page 100
💡 Result:
The GitHub CLI command gh run list provides a --limit flag (alias -L) to control the number of workflow runs to fetch [1][2][3]. While the specific default value for the --limit flag may vary across versions of the GitHub CLI (commonly cited as 20 or 30) [1][2][4][3], the underlying GitHub Actions REST API uses a per_page parameter to handle pagination [5][6][7]. Key points regarding these parameters include: 1. GitHub CLI --limit: This flag specifies the maximum number of runs to retrieve [1][8]. If not specified, it defaults to a built-in value (often 20 in recent versions) [1][2][3]. 2. GitHub Actions API per_page: This API parameter controls the number of results returned per request [5][6]. It has a default value of 30 and a maximum allowed value of 100 [5][6][9]. 3. Interaction: When you use gh run list --limit, the GitHub CLI manages the API calls for you, performing internal pagination if the requested limit exceeds the API's per_page maximum (100) or to aggregate results to reach your specified limit [4]. If you need to retrieve a large number of runs, the GitHub CLI will handle the necessary API pagination to fulfill your --limit request [4]. If you are working directly with the GitHub REST API, you must explicitly set per_page (up to 100) and iterate through pages using the page parameter to retrieve more than 100 results [5][6][7].
Citations:
- 1: https://cli.github.com/manual/gh_run_list
- 2: https://manpages.debian.org/testing/gh/gh-run-list.1.en.html
- 3: gh run list use of status is confusing blend of status and conclusion cli/cli#8106
- 4: gh run view limit on output? cli/cli#7771
- 5: https://docs.github.com/en/rest/actions/workflow-runs?apiVersion=2026-03-10
- 6: https://docs.github.com/en/enterprise-cloud@latest/rest/actions/workflow-runs?apiVersion=2022-11-28
- 7: https://docs.github.com/en/enterprise-server@3.19/rest/actions/workflow-runs
- 8: https://fossies.org/linux/gh-cli/pkg/cmd/run/list/list.go
- 9: Add missing params on get_workflow_runs() PyGithub/PyGithub#2705
🌐 Web query:
site:docs.github.com REST API actions workflow runs list workflow runs filtered by created
💡 Result:
To filter workflow runs by their creation date using the GitHub REST API, you use the created parameter [1][2]. This parameter is available for the endpoints that list workflow runs for a repository and for a specific workflow [1][3]. The created parameter accepts a string that defines a date-time range [1][4]. The syntax for this string follows the standard GitHub search syntax, which allows for various operators such as ranges [1][5]. For example, you can specify dates in YYYY-MM-DD format or ISO 8601 date-time formats [1][4]. Applicable API Endpoints: 1. List workflow runs for a repository: GET /repos/{owner}/{repo}/actions/runs [1][3] 2. List workflow runs for a workflow: GET /repos/{owner}/{repo}/actions/workflows/{workflow_id}/runs [1][3] When using the created parameter, the API endpoint will return up to 1,000 results for the query [1][2]. For detailed information on the specific operators supported for date ranges (e.g., >, <,..), refer to the official GitHub documentation on search syntax [1][4].
Citations:
- 1: https://docs.github.com/en/rest/actions/workflow-runs?apiVersion=2026-03-10
- 2: https://docs.github.com/en/enterprise-cloud@latest/rest/actions/workflow-runs?apiVersion=2022-11-28
- 3: https://docs.github.com/en/enterprise-server@3.19/rest/actions/workflow-runs?apiVersion=2022-11-28
- 4: https://docs.github.com/enterprise-server@3.8/rest/actions/workflow-runs
- 5: https://docs.github.com/en/enterprise-server@3.19/rest/actions/workflow-runs
Fail closed when the run query reaches its cap.
--limit 500 stops this query after the 500 newest in-window runs, even though the API can return up to 1,000 matching created results. If more than 500 versions runs exist in the 8-day window, older failures are dropped before filtering and the workflow can report all-clear. Fetch a sentinel row and fail closed when it is present, or establish a guaranteed lower bound for this repository’s weekly Dependabot run count.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.github/workflows/dependabot-failure-watcher.yml around lines 93 - 98,
Update the run query in the workflow’s gh run list block to detect truncation by
requesting one sentinel row beyond the intended 500 results, then fail closed
when that sentinel is present before filtering or reporting success. Preserve
the existing in-window query and ensure the sentinel cannot be mistaken for a
real run result.
GitHub runs both Dependabot version updates and Dependabot security
updates under one workflow name, "Dependabot Updates", and the watcher
counted both. Security updates routinely fail for reasons no pull request
can fix -- the advisory is against a dependency this project does not
declare directly, or no patched version is reachable. Left alone the
watcher stays red every week on those and trains everyone to ignore it.
Filter those runs out by title. Version updates are unaffected.
The title check is subtler than it looks, so document it properly. A
security job is marked by "/." AND a " for " suffix together; version
updates are either "/." with no " for " (the scheduled scan) or "/" with
one (the pull request). Both halves of " in /. for " are therefore
load-bearing -- matching on " in /." alone would discard every scan run,
which is most of the version-update runs and the shape the failures this
watcher was written for actually took.
That "/." spelling only separates the two at the repo root. In a
subdirectory a security update and a version update's pull request render
identically, so drop " in /e2e/{js,ts} for " by name as well. The Node
repos this workflow is shared with carry committed lockfiles under e2e/js
and e2e/ts, whose transitive dev dependencies attract advisories no pull
request can fix, and nothing in either is shipped code. That is 16
unactionable failures in each of GeoIP2-node and minfraud-api-node over
retained history; repos without those directories are unaffected.
Unlike the root filter, this one is not free. Both Node repos configure
npm with directories: ["/", "**/*"], and that glob does match e2e/js and
e2e/ts, so those directories do get version updates -- there is an open
version-update pull request under e2e/ts in both repos as this is
written. Dropping the pattern discards their pull-request refresh
failures along with the security jobs, and the ecosystem label is no help
because Dependabot writes "npm_and_yarn" for both. Taken anyway: the
scheduled scan is what this watcher primarily exists to catch and is
still reported for those directories, so what is given up is the narrower
"one open pull request has gone stale" signal for two directories of test
scaffolding. After filtering, 4 genuine failures remain reported in
GeoIP2-node and 3 in minfraud-api-node.
Reading the directories out of dependabot.yml would look more general and
was the earlier plan here, but it fails green. Entries may use globs, and
minfraud-api-dotnet's directories: ["**/*"] yields titles like "nuget in
/**/*" for the scan and "nuget in /MaxMind.MinFraud for
System.Net.Http.Json" for the pull request, neither of which any literal
comparison against the configured value matches -- so its two real nuget
failures would have been dropped without a word. A stale denylist
re-introduces noise, which is loud; a stale allowlist hides failures.
Name the three kinds of run in the comment while here, because the
scheduled scan and the per-pull-request refresh are easy to conflate: the
refresh runs are one per open pull request and are triggered by pushes to
the base branch or by rebases, not by the schedule, so they arrive in
bursts after merges. The scan is the kind this watcher primarily exists
to catch, which is what makes hiding a hypothetical refresh failure under
e2e an acceptable cost rather than a hole.
Bound the query server-side with --created instead of fetching all of
history and filtering by date locally, so --limit now caps an
already-narrowed window rather than standing in for one, and the run list
drops from several API pages to one. --limit rises 100 -> 500 as a
backstop: it still applies before the title filter, and reaching it would
silently drop the oldest in-window runs.
This workflow is shared verbatim across MaxMind repos. The change was
developed in maxmind/device-android and is applied here unmodified; see
that repo's commit for the measurements it was derived from.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
c9f80e3 to
1da936c
Compare
There was a problem hiding this comment.
♻️ Duplicate comments (1)
.github/workflows/dependabot-failure-watcher.yml (1)
99-109:⚠️ Potential issue | 🟠 MajorThe 500-run cap still makes the watcher fail open.
If the query contains 500 or more runs, older runs are omitted before
jqevaluates failures, so the workflow can report all-clear while an omitted run is failing.gh run listdocuments--limitas the maximum number of runs to fetch. (cli.github.com)Fetch one sentinel row beyond the intended limit and fail closed when it is present, or establish a guaranteed repository-specific upper bound. This is the same unresolved issue raised in the previous review.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In @.github/workflows/dependabot-failure-watcher.yml around lines 99 - 109, Update the run query in the watcher step so it cannot report all-clear when the result set exceeds the inspected window: fetch one sentinel row beyond the intended limit and detect its presence before evaluating failures, failing closed when present. Preserve the existing repository, workflow, date filtering, and jq failure evaluation.Source: MCP tools
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Duplicate comments:
In @.github/workflows/dependabot-failure-watcher.yml:
- Around line 99-109: Update the run query in the watcher step so it cannot
report all-clear when the result set exceeds the inspected window: fetch one
sentinel row beyond the intended limit and detect its presence before evaluating
failures, failing closed when present. Preserve the existing repository,
workflow, date filtering, and jq failure evaluation.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 9b24386d-298c-48a3-a78f-4dc509a52dc4
📒 Files selected for processing (1)
.github/workflows/dependabot-failure-watcher.yml
GitHub runs Dependabot version updates and Dependabot security updates under one
workflow name,
Dependabot Updates, and the watcher counted both. Securityupdates routinely fail for reasons no pull request can fix -- the advisory is
against a dependency the project does not declare directly, or no patched
version is reachable -- so the watcher stays red every week on those and trains
everyone to ignore it.
This filters security-update runs out by title, documents why both halves of
" in /. for "are load-bearing, and bounds thegh run listquery server-sidewith
--createdinstead of fetching all of history and filtering locally.See the commit message for the full reasoning, including why reading the
directory list out of
dependabot.ymlwas tried and rejected.This workflow is shared verbatim across MaxMind repos. The change was developed
in maxmind/device-android (maxmind/device-android#71)
and is applied here unmodified; the resulting file is byte-identical in every
repo.
Summary by CodeRabbit
Bug Fixes
Documentation