Fix: derive the reported status from job-results - #1
Merged
Conversation
The status in the notification was never derived from `job-results`. The parser word-split the input and matched each word against `^([^:]+):([^:]+)$`, so any comma-separated list arrived as a single multi-colon word, matched nothing, and produced `{}`. Every probe against `{}` is false, so the code fell through to its `else` branch and reported SUCCESS. Both C++ build repos passed commas, so wire-sysio and wire-cdt posted `✅ SUCCESS` over failing builds — wire-sysio run 34144933071 failed its gcc leg and still reported green.
Accept commas alongside spaces and newlines, and invert the status derivation to fail closed: SUCCESS only when at least one entry parsed and every one is exactly `success`, anything unrecognised is UNKNOWN, and input that parses to nothing is a hard error. Under the old ordering any parsing defect degraded silently to green, which is why this went unnoticed.
Three further defects fixed:
`JOB_DETAILS` was accumulated in a `while read` on the right of a pipe, so every append landed in a subshell and was discarded. The per-job breakdown never rendered for anyone, including the callers whose single-pair input parsed correctly.
The documented JSON form never worked. Inputs were interpolated unquoted into a double-quoted shell assignment, which strips the inner quotes and leaves invalid JSON that failed the same way — and left workflow names and branch names able to break the payload or the script. Inputs now cross into the script through the environment, and the payload is assembled with `jq --arg` instead of string concatenation.
Errors were raised from inside a command substitution, where `exit` ends only the subshell and the `::error::` line is captured into the caller's variable rather than logged. Failures surfaced as a step that died with no stated reason.
The implementation moves to `scripts/notify.sh` so it can be tested; `action.yaml` now only maps inputs to the environment. `tests/run-tests.sh` covers every separator form, the status precedence, the fail-closed paths, the rendered payload, and the webhook call itself against a local sink; each of the four defects above fails the suite when reintroduced.
Existing callers are unaffected: every current `job-results` value, comma-separated included, was replayed against the new script and reports the same status a correct implementation would.
README: correct three `uses:` paths that named a repository and workflow path that do not exist, replace `notification-type: mattermost` with the `1` the code actually compares against, and document the separators and the fail-closed contract.
This was referenced Sep 7, 2026
jglanz
approved these changes
Sep 8, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The status in the notification was never derived from
job-results. The parser word-split the input and matched each word against^([^:]+):([^:]+)$, so any comma-separated list arrived as a single multi-colon word, matched nothing, and produced{}. Every probe against{}is false, so the code fell through to itselsebranch and reported SUCCESS.Both C++ build repos passed commas, so wire-sysio and wire-cdt have been posting
✅ SUCCESSover failing builds. wire-sysio run 34144933071 failed its gcc build leg and reported green; its log shows the parse landing onResult: {}andOverall: SUCCESS. Merge gates were never affected — the runs showed red on GitHub, only the chat message was wrong.What changed
Commas are accepted alongside spaces and newlines, and the status derivation is inverted to fail closed: SUCCESS only when at least one entry parsed and every one is exactly
success, anything unrecognised is UNKNOWN, and input that parses to nothing is a hard error. Under the old ordering any parsing defect degraded silently to green, which is why this went unnoticed for months.Three further defects, all of which predate this PR:
JOB_DETAILSwas accumulated in awhile readon the right of a pipe, so every append landed in a subshell and was discarded. The per-job breakdown never rendered for anyone, including the callers whose single-pair input parsed correctly.jq --argrather than string concatenation.exitends only the subshell and the::error::line is captured into the caller's variable instead of being logged. Failures surfaced as a step that died without stating a reason.Structure
The implementation moves to
scripts/notify.shso it can be exercised directly;action.yamlnow only maps inputs to the environment.NOTIFY_DRY_RUN=1prints the payload and skips the webhook.tests/run-tests.shcovers each separator form, the status precedence, the fail-closed paths, the rendered payload, and the webhook call itself against a local sink that rejects anything which is not well-formed JSON. Each of the four defects above was reintroduced individually to confirm the suite catches it rather than merely passing alongside it.Compatibility
Every current caller's
job-resultsvalue — wire-docs, wire-ethereum (x2), eth-validator-deposit-ui, wire-cdt (x2) and wire-sysio, comma-separated forms included — was replayed against the new script and reports what a correct implementation would. The comma callers are fixed by this PR alone, independently of the workflow changes in Wire-Network/wire-sysio#602 and Wire-Network/wire-cdt#116.Callers pinned to
@v1pick this up only once the tag is moved.README
Three
uses:paths named a repository and workflow path that do not exist.notification-type: mattermostappeared in two examples where the code compares against1and2. Both corrected, and the separators and fail-closed contract are now documented.