From c1a461410862f1faa9721cec633bb5307346661e Mon Sep 17 00:00:00 2001 From: n Date: Wed, 5 Aug 2026 14:48:26 -0500 Subject: [PATCH 1/4] Watch CI with gh instead of a polling loop The CI watcher was told to poll for check runs every thirty seconds. A foreground sleep does not run in this harness and the agent has no timer, so the interval it was asked for was one it could not take. The documented way out was gh pr checks --watch, and gh was not installed, so that path was dead too. gh is now a hard dependency rather than an alternative offered where it happens to exist. Watching is a single blocking --watch call, and the exit code distinguishes a failure from a watch cut short before the round finished, which would otherwise burn a fix attempt on a run that was merely still going. Failure diagnosis reads the failing job's log through gh run view --log-failed rather than scraping check run annotations. Annotations carry compiler diagnostics well and test failures poorly, and the report this agent writes is the only thing the implementer gets. gh carries its own credential, separate from the MCP server's, so the agent checks gh auth status first and names the missing login rather than failing part way through with an authentication error. crypter-triage-review named gh pr comment as the only way to answer a finding with no thread, which was unreachable for the same reason. It now names the MCP equivalent alongside it. Co-Authored-By: Claude Opus 5 --- .claude/agents/ci-watcher.md | 58 +++++++++++++++---- .claude/skills/crypter-triage-review/SKILL.md | 7 ++- 2 files changed, 53 insertions(+), 12 deletions(-) diff --git a/.claude/agents/ci-watcher.md b/.claude/agents/ci-watcher.md index e8eac9bf..dbdb9b25 100644 --- a/.claude/agents/ci-watcher.md +++ b/.claude/agents/ci-watcher.md @@ -24,9 +24,19 @@ The pull request is on the repository the branch was pushed to: git -C remote get-url origin ``` -Use `mcp__github__pull_request_read` with `method: "get_check_runs"` for the head commit's -checks. Where the `gh` CLI is installed, `gh pr checks --watch` and `gh run list --commit ` -followed by `gh run view --log-failed` give more detail; use them when they are there. +The `gh` CLI is what you watch and diagnose with. `mcp__github__pull_request_read` is there for +structured pull request data when you want it. Do not reach for the REST API directly. + +`gh` needs its own credential, separate from the one the MCP server holds. Confirm it before you +start: + +```bash +gh auth status +``` + +**If it reports no logged-in host, stop and say that `gh auth login` has not been run.** Say it +as the setup problem it is. Every command below fails without it, and failing at the first one +with an authentication error looks nothing like the real answer, which is that nobody logged in. ## Find the run @@ -40,14 +50,31 @@ Confirm you are reading the round for the commit you were asked about: git -C rev-parse ``` -Compare that against the head SHA in the pull request data. A push takes a moment to register, -so poll `get_check_runs` until runs appear. If nothing has appeared after a few minutes, say so -and stop: a push that starts no checks is a setup problem no amount of waiting fixes. +Compare that against the head SHA the pull request reports. A push takes a moment to register, +so a check run may not exist the instant you are invoked. + +`gh pr checks` says `no checks reported on the ... branch` when none have started. That is the +one case worth distinguishing from a failure: if it still says that after a few minutes, stop +and say so, because a push that starts no checks is a setup problem no amount of waiting fixes. ## Watch -Poll until every check reaches a conclusion. Give it a generous timeout — a full build plus the -test suite is slow, and a watch you cut short looks exactly like a failure. +```bash +gh pr checks --repo / --watch +``` + +`--watch` blocks until every check reaches a conclusion, so this is one call rather than a +polling loop. **Never write a polling loop with `sleep` in it** — a foreground `sleep` does not +run here, and `--watch` exists precisely so you do not need one. + +Give the call a long timeout. A full round is several minutes, and the tool caps at ten. If the +call is killed before the round finishes, run it again — a watch you cut short looks exactly +like a failure. + +The exit code is your signal, and it separates the two outcomes you would otherwise confuse: +`0` when everything passed, `1` when something failed, `8` when checks are still pending. An `8` +after `--watch` means the call was cut short rather than that CI is unhappy — run it again +rather than reporting a failure. Five workflows run on a pull request, reported by job name rather than by workflow name. Expect these: @@ -72,8 +99,19 @@ not have caught locally shows up. ## On failure -Get the real error. The check run's `output` summary and annotations carry the diagnostic; -where `gh` is installed, the failed job's log carries more. +Get the real error out of the failing job's log: + +```bash +gh run list --commit --repo / +gh run view --repo / --log-failed +``` + +`--log-failed` prints only the steps that failed, which is usually the whole diagnosis — the +compiler diagnostic, the assertion message, the analyzer rule. Where a log is too long to read +whole, grep it rather than skimming it. + +Where the log still leaves you short of the cause, say so in the report and give the run URL. Do +not fill the gap by guessing at a cause the log does not support. Then read the code the failure points at. The repository's working tree is on whatever the user last checked out, so read the branch's version: diff --git a/.claude/skills/crypter-triage-review/SKILL.md b/.claude/skills/crypter-triage-review/SKILL.md index 6db73960..2f36a82c 100644 --- a/.claude/skills/crypter-triage-review/SKILL.md +++ b/.claude/skills/crypter-triage-review/SKILL.md @@ -72,8 +72,11 @@ summary. Every finding gets one of three outcomes, and none of them is silence. -**Does not hold** — reply on the thread with `add_reply_to_pull_request_comment`, or -`gh pr comment` where the finding has no thread. Give the evidence: what the code does instead, +**Does not hold** — reply on the thread with `add_reply_to_pull_request_comment`. Where the +finding has no thread to reply on, comment on the pull request itself with `add_issue_comment` +or `gh pr comment`, quoting enough of the finding that the reply stands on its own. + +Give the evidence: what the code does instead, by file and line. Two or three sentences. Say it as a position, not a verdict — the person who raised it may know something the verifier could not see, and the thread is where that comes out. From a5fd0b08e1c5d741f61b0011b3402a84d8f1e39f Mon Sep 17 00:00:00 2001 From: n Date: Wed, 5 Aug 2026 14:56:00 -0500 Subject: [PATCH 2/4] Treat a gh authentication failure as a setup problem An unauthenticated gh pr checks exits 4, which was outside the set of exit codes the watcher was given. Left undocumented it reads as a non-zero exit and therefore as a failing build, which sends the implementer looking for a defect that is not there. Co-Authored-By: Claude Opus 5 --- .claude/agents/ci-watcher.md | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-) diff --git a/.claude/agents/ci-watcher.md b/.claude/agents/ci-watcher.md index dbdb9b25..03e614c5 100644 --- a/.claude/agents/ci-watcher.md +++ b/.claude/agents/ci-watcher.md @@ -71,10 +71,18 @@ Give the call a long timeout. A full round is several minutes, and the tool caps call is killed before the round finishes, run it again — a watch you cut short looks exactly like a failure. -The exit code is your signal, and it separates the two outcomes you would otherwise confuse: -`0` when everything passed, `1` when something failed, `8` when checks are still pending. An `8` -after `--watch` means the call was cut short rather than that CI is unhappy — run it again -rather than reporting a failure. +The exit code is your signal, and it separates outcomes you would otherwise confuse: + +| Exit | Means | +|---|---| +| `0` | Every check passed | +| `1` | A check failed | +| `4` | `gh` is not authenticated — a setup problem, not a CI result | +| `8` | Checks are still pending | + +An `8` after `--watch` means the call was cut short rather than that CI is unhappy; run it again +rather than reporting a failure. A `4` means nobody ran `gh auth login`, and reporting that as a +failing build would send the implementer hunting for a defect that does not exist. Five workflows run on a pull request, reported by job name rather than by workflow name. Expect these: From ea721ba49fdc4d4383521ad15bda4893b6598584 Mon Sep 17 00:00:00 2001 From: n Date: Wed, 5 Aug 2026 15:17:13 -0500 Subject: [PATCH 3/4] Move the CI watcher's mechanics into a script Waiting, reading exit codes, finding failing runs and cutting a log down to the part that explains itself are all deterministic, and they were written as prose for an agent to follow. Prose cannot be run, so the mistakes in it only surfaced when a run went wrong: a polling interval the harness cannot take, an exit code that reads as a failing build when it means nobody logged in, an instruction to read the tail of a log whose last fifty lines are cleanup. ci-status.sh does that half and can be tested. Exit codes separate a CI failure from the three setup problems that otherwise look like one. On a failure it prints the window of log ending at the runner's ##[error] marker, which turns a 1100-line dump into roughly sixty lines containing the cause. The agent keeps the half that needs judgement: reading back from the symptom to the cause, reading the branch's version of the code the failure names, deciding whether it is a defect, a wrong test or the plan itself being wrong. Co-Authored-By: Claude Opus 5 --- .claude/agents/ci-watcher.md | 93 +++++++++----------------- .claude/scripts/ci-status.sh | 122 +++++++++++++++++++++++++++++++++++ 2 files changed, 153 insertions(+), 62 deletions(-) create mode 100755 .claude/scripts/ci-status.sh diff --git a/.claude/agents/ci-watcher.md b/.claude/agents/ci-watcher.md index 03e614c5..94e16a88 100644 --- a/.claude/agents/ci-watcher.md +++ b/.claude/agents/ci-watcher.md @@ -24,65 +24,31 @@ The pull request is on the repository the branch was pushed to: git -C remote get-url origin ``` -The `gh` CLI is what you watch and diagnose with. `mcp__github__pull_request_read` is there for -structured pull request data when you want it. Do not reach for the REST API directly. - -`gh` needs its own credential, separate from the one the MCP server holds. Confirm it before you -start: - -```bash -gh auth status -``` - -**If it reports no logged-in host, stop and say that `gh auth login` has not been run.** Say it -as the setup problem it is. Every command below fails without it, and failing at the first one -with an authentication error looks nothing like the real answer, which is that nobody logged in. - -## Find the run - -The pull request is a draft and stays one; the user takes it out of draft when they are ready -to review it. Checks run on drafts, so pushing the branch starts a round of them, and a round -is already queued or finished by the time you are invoked. - -Confirm you are reading the round for the commit you were asked about: - -```bash -git -C rev-parse -``` - -Compare that against the head SHA the pull request reports. A push takes a moment to register, -so a check run may not exist the instant you are invoked. - -`gh pr checks` says `no checks reported on the ... branch` when none have started. That is the -one case worth distinguishing from a failure: if it still says that after a few minutes, stop -and say so, because a push that starts no checks is a setup problem no amount of waiting fixes. - ## Watch +`.claude/scripts/ci-status.sh` does the waiting and the log archaeology. Run it and read what it +gives you: + ```bash -gh pr checks --repo / --watch +.claude/scripts/ci-status.sh / ``` -`--watch` blocks until every check reaches a conclusion, so this is one call rather than a -polling loop. **Never write a polling loop with `sleep` in it** — a foreground `sleep` does not -run here, and `--watch` exists precisely so you do not need one. +It blocks until every check concludes, so **never write a polling loop with `sleep` in it** — a +foreground `sleep` does not run here. Give the call a long timeout; a full round is several +minutes and the tool caps at ten. -Give the call a long timeout. A full round is several minutes, and the tool caps at ten. If the -call is killed before the round finishes, run it again — a watch you cut short looks exactly -like a failure. +Its exit code is the outcome, and it separates cases you would otherwise confuse: -The exit code is your signal, and it separates outcomes you would otherwise confuse: +| Exit | Means | What to do | +|---|---|---| +| `0` | Every check passed | Report success | +| `1` | A check failed | The log extract is on stdout; diagnose it | +| `3` | No checks ever started | A setup problem. Stop and say so | +| `4` | `gh` is not authenticated | A setup problem. Say `gh auth login` has not been run | +| `8` | Still pending when the watch ended | The call was cut short. Run it again | -| Exit | Means | -|---|---| -| `0` | Every check passed | -| `1` | A check failed | -| `4` | `gh` is not authenticated — a setup problem, not a CI result | -| `8` | Checks are still pending | - -An `8` after `--watch` means the call was cut short rather than that CI is unhappy; run it again -rather than reporting a failure. A `4` means nobody ran `gh auth login`, and reporting that as a -failing build would send the implementer hunting for a defect that does not exist. +`3`, `4` and `8` are **not** CI failures. Reporting any of them as one sends an implementer +hunting for a defect that does not exist. Five workflows run on a pull request, reported by job name rather than by workflow name. Expect these: @@ -92,8 +58,7 @@ these: | `changes / detect` | Never. Every workflow gates on `detect-code-changes`, so there are five of these. | | `build-and-test` | The diff is documentation only | | `build-and-test-web` | The diff is documentation only | -| `Analyze (csharp)` | The diff is documentation only | -| `Analyze (javascript)` | The diff is documentation only | +| `Analyze (csharp)` and `Analyze (javascript)` | The diff is documentation only | | `build-api` | The diff is documentation only | | `build-web` | The diff is documentation only | | `build-devcontainer` | The diff does not touch `.devcontainer/` | @@ -107,19 +72,23 @@ not have caught locally shows up. ## On failure -Get the real error out of the failing job's log: +The script has already found the failing runs and printed the window of log ending at the +runner's `##[error]` marker. That window is where the diagnosis is, and reading it is the job. + +**The marker line is the symptom, not the cause.** It says things like `buildx failed with: +ERROR: ... exit code: 1`. The thing that actually broke — a version mismatch, a compiler +diagnostic, a failing assertion — sits in the lines above it. Work upwards until you find +something that explains the failure rather than restating it. + +Where the extract leaves you short of the cause, pull more of the log yourself rather than +guessing: ```bash -gh run list --commit --repo / -gh run view --repo / --log-failed +gh run view --repo / --log-failed > ``` -`--log-failed` prints only the steps that failed, which is usually the whole diagnosis — the -compiler diagnostic, the assertion message, the analyzer rule. Where a log is too long to read -whole, grep it rather than skimming it. - -Where the log still leaves you short of the cause, say so in the report and give the run URL. Do -not fill the gap by guessing at a cause the log does not support. +Say so in the report if it still does not explain the failure, and give the run URL. Do not fill +the gap with a cause the log does not support. Then read the code the failure points at. The repository's working tree is on whatever the user last checked out, so read the branch's version: diff --git a/.claude/scripts/ci-status.sh b/.claude/scripts/ci-status.sh new file mode 100755 index 00000000..3e5b69ac --- /dev/null +++ b/.claude/scripts/ci-status.sh @@ -0,0 +1,122 @@ +#!/usr/bin/env bash +# Watch a pull request's checks to a conclusion and, where they failed, print the part of the +# log that explains why. +# +# This is the mechanical half of watching CI: waiting, reading exit codes, finding the failing +# runs, and cutting the noise out of their logs. Deciding what the error means belongs to +# whoever reads the output. +# +# usage: ci-status.sh {pr-number} [owner/repo] +# +# Exit codes are the caller's signal and are deliberately distinct: +# 0 every check passed +# 1 a check failed; the log extract is on stdout +# 3 no checks have started for the head commit +# 4 gh is not authenticated +# 8 checks were still pending when the watch ended +set -uo pipefail + +pr_number="${1:-}" +repo="${2:-}" + +if [[ -z "${pr_number}" ]]; then + echo "usage: ci-status.sh {pr-number} [owner/repo]" >&2 + exit 64 +fi + +if ! gh auth status >/dev/null 2>&1; then + echo "gh is not authenticated. Run 'gh auth login'." >&2 + exit 4 +fi + +gh_args=(--repo "${repo}") +[[ -z "${repo}" ]] && gh_args=() + +head_sha=$(gh pr view "${pr_number}" "${gh_args[@]}" --json headRefOid --jq .headRefOid 2>/dev/null) +if [[ -z "${head_sha}" ]]; then + echo "Could not read pull request ${pr_number}." >&2 + exit 64 +fi + +echo "Head commit: ${head_sha}" + +checks=$(gh pr checks "${pr_number}" "${gh_args[@]}" --watch 2>&1) +watch_status=$? + +# A pull request whose checks never started reports this rather than an empty table, and it +# means a setup problem rather than a slow queue. +if grep -qi "no checks reported" <<<"${checks}"; then + echo "No checks have started for ${head_sha}." + exit 3 +fi + +echo "${checks}" + +case "${watch_status}" in + 0) + echo + echo "All checks passed." + exit 0 + ;; + 8) + echo + echo "Checks still pending when the watch ended. Run again." + exit 8 + ;; +esac + +# Strip the "jobsteptimestamp " prefix gh puts on every log line, which is most of the +# width and none of the information. +strip_prefix() { + sed -E 's/^[^\t]*\t[^\t]*\t[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9:.]+Z //' +} + +# The runner marks the failure with ##[error], but that line is the symptom — "the build +# failed". The cause sits above it, so print a window ending at the marker. +extract_failure() { + local log="$1" + local marker + marker=$(grep -n '##\[error\]' "${log}" | head -1 | cut -d: -f1) + + if [[ -z "${marker}" ]]; then + echo " No ##[error] marker found. Last 40 lines:" + tail -40 "${log}" | strip_prefix | sed 's/^/ /' + return + fi + + local from=$(( marker - 60 )) + (( from < 1 )) && from=1 + sed -n "${from},$(( marker + 2 ))p" "${log}" | strip_prefix | sed 's/^/ /' +} + +echo +echo "=== Failing runs for ${head_sha} ===" + +failed_runs=$(gh run list --commit "${head_sha}" "${gh_args[@]}" \ + --status failure --json databaseId,workflowName --jq '.[] | "\(.databaseId)\t\(.workflowName)"') + +if [[ -z "${failed_runs}" ]]; then + echo "A check failed but no failing workflow run was found for the commit." + echo "The failure may belong to a check that is not an Actions run." + exit 1 +fi + +log_dir=$(mktemp -d) +trap 'rm -rf "${log_dir}"' EXIT + +while IFS=$'\t' read -r run_id workflow_name; do + [[ -z "${run_id}" ]] && continue + echo + echo "--- ${workflow_name} (run ${run_id}) ---" + echo " https://github.com/${repo:-$(gh repo view --json nameWithOwner --jq .nameWithOwner)}/actions/runs/${run_id}" + echo + + log="${log_dir}/${run_id}.log" + if gh run view "${run_id}" "${gh_args[@]}" --log-failed >"${log}" 2>/dev/null && [[ -s "${log}" ]]; then + extract_failure "${log}" + else + echo " Could not read the failed log for run ${run_id}." + fi +done <<<"${failed_runs}" + +exit 1 From acd947d503bb143c26f143aebd829973b529f6ea Mon Sep 17 00:00:00 2001 From: n Date: Wed, 5 Aug 2026 16:14:52 -0500 Subject: [PATCH 4/4] Keep gh behind the script rather than in agent prose Two skills offered a bare gh call as an alternative to the MCP server for posting a review and commenting on a pull request. The MCP server does both, so those were discretionary CLI invocations with nothing to recommend them. The watcher's remaining direct call was fetching a fuller copy of a log the script had already downloaded. The script now keeps each failing job's log and prints its path, so reading further means opening a file rather than going back to the network. Nothing outside ci-status.sh invokes gh now. Co-Authored-By: Claude Opus 5 --- .claude/agents/ci-watcher.md | 9 +++------ .claude/scripts/ci-status.sh | 8 ++++++-- .claude/skills/crypter-review/SKILL.md | 2 +- .claude/skills/crypter-triage-review/SKILL.md | 4 ++-- 4 files changed, 12 insertions(+), 11 deletions(-) diff --git a/.claude/agents/ci-watcher.md b/.claude/agents/ci-watcher.md index 94e16a88..4e5bb322 100644 --- a/.claude/agents/ci-watcher.md +++ b/.claude/agents/ci-watcher.md @@ -80,12 +80,9 @@ ERROR: ... exit code: 1`. The thing that actually broke — a version mismatch, diagnostic, a failing assertion — sits in the lines above it. Work upwards until you find something that explains the failure rather than restating it. -Where the extract leaves you short of the cause, pull more of the log yourself rather than -guessing: - -```bash -gh run view --repo / --log-failed > -``` +Where the extract leaves you short of the cause, read further. The script keeps each failing +job's full log and prints its path, so open that file and search it rather than fetching another +copy. Say so in the report if it still does not explain the failure, and give the run URL. Do not fill the gap with a cause the log does not support. diff --git a/.claude/scripts/ci-status.sh b/.claude/scripts/ci-status.sh index 3e5b69ac..dc368d9d 100755 --- a/.claude/scripts/ci-status.sh +++ b/.claude/scripts/ci-status.sh @@ -101,8 +101,10 @@ if [[ -z "${failed_runs}" ]]; then exit 1 fi -log_dir=$(mktemp -d) -trap 'rm -rf "${log_dir}"' EXIT +# The full logs are kept rather than cleaned up. The extract below is a window, and whoever +# reads it may need more; leaving the files behind means they open a file instead of going back +# to the network for a second copy. +log_dir=$(mktemp -d -t ci-status-XXXXXX) while IFS=$'\t' read -r run_id workflow_name; do [[ -z "${run_id}" ]] && continue @@ -114,6 +116,8 @@ while IFS=$'\t' read -r run_id workflow_name; do log="${log_dir}/${run_id}.log" if gh run view "${run_id}" "${gh_args[@]}" --log-failed >"${log}" 2>/dev/null && [[ -s "${log}" ]]; then extract_failure "${log}" + echo + echo " Full log: ${log}" else echo " Could not read the failed log for run ${run_id}." fi diff --git a/.claude/skills/crypter-review/SKILL.md b/.claude/skills/crypter-review/SKILL.md index d009c06c..2d547481 100644 --- a/.claude/skills/crypter-review/SKILL.md +++ b/.claude/skills/crypter-review/SKILL.md @@ -84,7 +84,7 @@ this pull request may not be theirs. Use the GitHub MCP server's `pull_request_review_write` with method `create` to open a pending review, `add_comment_to_pending_review` for each finding that names a file and a line **in the -diff**, then `submit_pending`. Where `gh` is installed, `gh pr review --comment` posts the body. +diff**, then `submit_pending`. The review body carries: diff --git a/.claude/skills/crypter-triage-review/SKILL.md b/.claude/skills/crypter-triage-review/SKILL.md index 2f36a82c..d390d577 100644 --- a/.claude/skills/crypter-triage-review/SKILL.md +++ b/.claude/skills/crypter-triage-review/SKILL.md @@ -73,8 +73,8 @@ summary. Every finding gets one of three outcomes, and none of them is silence. **Does not hold** — reply on the thread with `add_reply_to_pull_request_comment`. Where the -finding has no thread to reply on, comment on the pull request itself with `add_issue_comment` -or `gh pr comment`, quoting enough of the finding that the reply stands on its own. +finding has no thread to reply on, comment on the pull request itself with `add_issue_comment`, +quoting enough of the finding that the reply stands on its own. Give the evidence: what the code does instead, by file and line. Two or three sentences. Say it as a position, not a verdict — the person who