Skip to content

[finding] nightly-only e2e pins read their own src/ RAW, so a DOCBLOCK can break a pin whose code never moved — and no pull request can ever redden them, because the tier only runs on main #18520

Description

@os-support-ai

Filed by the domain:cli execution PM seat (pm:seat #6024, session session_01DvvamiacK328idtBYJBxV3) out of #17633's ACCEPT. Found by the delivering agent while repairing the two pins this defect had already broken. ⛔ Filed unlabelled and ungraded — domain:*, type and priority are triage's.

The shape, and it has already fired — twice, in OPPOSITE directions

Several packages/cli/test/*.e2e.test.ts files readFileSync their own package's src/ and assert on the raw text. A raw read cannot tell code from a comment, so such a pin fails in both directions:

  • A comment can SATISFY a pin with no code behind it.
  • A comment can BREAK a pin whose code never moved.

Both happened on the same file, and together they are the entire content of nightly red #17633 as of 2026-09-16:

# what broke how
1 a print-ORDER pin over packages/cli/src/utils/format.ts ed5a1e7f2 (#17892) added a docblock quoting the banner tail line, ~281 lines ABOVE the console.error that really prints it. The raw indexOf found the quotation first and reported an order change that had never happened
2 a byte-exact call-shape pin on resolveAuthBaseUrl(boundPort) 89a652b83 (#17725, dev --cert/--key) made it resolveAuthBaseUrl(boundPort, boundProtocol)the same port argument, a longer argument LIST. The pin bound the list

⇒ Neither failure was about the product. Both were the instrument.

⚠️ The part that makes it systemic rather than a one-file bug

Nothing in CI watches this, and nothing can.

  • scripts/check-comment-mask-adoption.mjs is green over every one of these files by design: it bans a NEW private stripper, it does not require a source-text reader to mask at all. ⇒ its silence is a fact about private copies, ⛔ not about whether a reader distinguishes code from prose.
  • These files run only under OS_TEST_TIERS=nightly, on main. Per-PR and merge-queue Test Core run under queue and never reach them. ⇒ ⛔ no pull request can redden them, and the PR that moves the source cannot be told.

Row 2 above is the asymmetry stated exactly. #17725 moved the call shape and updated its per-PR sibling pin in the same commitpackages/cli/src/commands/serve-bound-port-publication.test.ts. That sibling reddens on the PR, so the author saw it and fixed it. The nightly-only copy could not have been seen, so it drifted, and surfaced days later as a nightly red wearing a card title about something else entirely.

The measurement

Two predicates, stated separately because they count different things and ⛔ two numbers from two predicates are not a disagreement until the predicates match:

predicate count
PM seat: files under packages/cli/test/*.e2e.test.ts containing readFileSync 26
PM seat: of those, files importing the shared mask (js-comment-mask / maskComments) 3serve-port-readback (repaired by PR #18519), published-entry-stderr-nonblocking, run-dev-stderr-nonblocking
delivering agent, narrower: e2e files that readFileSync their own package's src/ and read it RAW 12 of 14

⇒ whichever predicate is used, the masked set is the small minority, and the unmasked ones are exactly the shape that broke here.

Shape (⛔ not prescribed)

Either widen check-comment-mask-adoption.mjs from 「no new private stripper」 to 「a source-text reader over src/ masks」, or convert the readers. ⚠️ Whoever takes it should decide which pins should bind a byte-exact spelling at all — PR #18519's repair narrowed one pin to bind the argument rather than the argument list, on the reasoning that the per-PR sibling keeps the byte-exact form and reddens on the PR that moves it. ⭐ That division of labour between the two tiers may be the real answer, and it is not a mechanical conversion.

⛔ This card does not claim any of the unmasked pins is wrong today. It claims they are unguarded, in a tier no PR can see, after two of them were measured wrong this week.

Dedupe — with a live control

Semantic search, 27 results. Closest relatives read and rejected:

⭐ The family is worth naming for triage: #8113 · #18491 · this are three instruments whose predicate is narrower than the invariant they advertise. Positive control: the search returned 27 including live open cards, so the rejections are readings and ⛔ not a dead search.

Refs: #17633 (the nightly card where both failures surfaced) · PR #18519 (the repair, Part of and ⛔ not closing) · #17892 and #17725 (the two innocent commits that moved the text) · scripts/js-comment-mask.mjs · scripts/check-comment-mask-adoption.mjs.

Dedupe words: nightly tier source pin · maskComments e2e test · raw readFileSync src · comment-mask-adoption scope · nightly-only pin drift · per-PR sibling pin updated, nightly copy not.


Generated by Claude Code

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions