Measured on dossiq run 34741589503, Quality Report job 103682603397, a development PR whose diff the path filter classified as code.
Three lines from the same job, in order:
PATH_FILTER_REASON: the diff touches code paths (1 file(s) matched the code globs).
...
Test-tier verdict census:
- Playwright: ENABLED, and deliberately NOT RUN — E2E runs locally and on the promotion path only. This pull request targets `development`, so the suite is asked once per promotion into `beta` and `main` ...
##[notice]Partial run — Playwright did not execute because the changed-path filter found nothing for them to decide. This run is green about the files it examined, not about the test suite.
The census line is correct: Playwright was skipped by e2e-promotion-only (#768), and the reason is recorded. The notice directly below it is wrong: the path filter did not find nothing, it matched a code file, and it is not why Playwright was skipped.
Why it matters. The notice is the line a reader sees in the job summary. On every development PR from now on it will tell them the path filter skipped E2E, on diffs where the path filter said the opposite. That is an instrument naming the wrong cause, which is worse than no notice: the next person who needs to know why E2E did not run will be sent to the wrong config.
The gates themselves are right. Gate — a test job that was ENABLED but never ran has no verdict passed on this run, so the aggregator correctly treats a promotion-only skip as deliberate rather than as a missing verdict. This is only the human-facing notice.
Fix. The notice has one message for "Playwright skipped" and assumes the path filter. It should branch on the same e2e / e2e_reason outputs the census already reads from the changes job, and print the recorded reason. Where the path filter genuinely was the cause, the existing wording stays.
Found while probing #768 on live traffic (dossiq#2626). Not fixed here.
Measured on dossiq run
34741589503, Quality Report job103682603397, a development PR whose diff the path filter classified as code.Three lines from the same job, in order:
The census line is correct: Playwright was skipped by
e2e-promotion-only(#768), and the reason is recorded. The notice directly below it is wrong: the path filter did not find nothing, it matched a code file, and it is not why Playwright was skipped.Why it matters. The notice is the line a reader sees in the job summary. On every development PR from now on it will tell them the path filter skipped E2E, on diffs where the path filter said the opposite. That is an instrument naming the wrong cause, which is worse than no notice: the next person who needs to know why E2E did not run will be sent to the wrong config.
The gates themselves are right.
Gate — a test job that was ENABLED but never ran has no verdictpassed on this run, so the aggregator correctly treats a promotion-only skip as deliberate rather than as a missing verdict. This is only the human-facing notice.Fix. The notice has one message for "Playwright skipped" and assumes the path filter. It should branch on the same
e2e/e2e_reasonoutputs the census already reads from thechangesjob, and print the recorded reason. Where the path filter genuinely was the cause, the existing wording stays.Found while probing #768 on live traffic (dossiq#2626). Not fixed here.