docs(docs-audit): state the recall denominator in both headers - #13600
Merged
os-project-manager merged 1 commit intoAug 31, 2026
Conversation
#13306 (maintainer ruling 2026-08-31): a recall figure for affected-docs.mjs must use the corpus it claims to cover — hand-written docs, excluding content/docs/references/** — as its denominator. Excluding generated pages is constructive design, not an omission: they are never hand-edited, so prompting an author about one is wrong advice. The one figure ever computed against a wider denominator (all content/docs edits) was measuring how often the docs generator ran in the sampling window, not this tool's recall (31 of its 46 ground-truth entries were generated pages the tool can never list). Adds that statement to affected-docs.mjs's own header (the authoritative half) and a short cross-reference in docs-drift-check.yml's header (which already documents the separate, page-level structural-invisibility class from #11356/#11357) so the two halves compose instead of each stating their own thing. No behavior change; no new recall figure computed.
Contributor
📓 Docs Drift CheckNothing in this diff resolved to a documentable surface (no symbol, route or SDK anchor derived from 0 changed package(s)), so this run has no opinion about the docs. What this run could not see
Coarse fallback — 0 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): |
os-project-manager
marked this pull request as ready for review
August 31, 2026 03:04
os-project-manager
deleted the
claude/issue-13306-recall-denominator-header
branch
August 31, 2026 03:23
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #13306
What
Writes the recall-denominator definition into both places a reader would look for it, per the maintainer's 2026-08-31 ruling in #13306:
scripts/docs-audit/affected-docs.mjs's header (the authoritative half, right after the existingScope:line).github/workflows/docs-drift-check.yml's header (a short cross-reference, since it already documents a separate structural-invisibility class from [finding] docs-drift 对「扩充可枚举词汇」的 PR 恒报绿 —— 页面上逐项枚举该词汇的表格随之变得不完整,而符号锚点看不见它。本班两次由人工判据抓出 #11356/finding: READMEs are still anchorless to docs-drift after #9282's close, so a README-staleness PR gets "no opinion" — two such cards landed today #11357 and previously said nothing about the denominator)No behavior change. No new recall figure is computed by this PR — the one figure cited is the already-measured, dated reading from #13306's own investigation (31 of 46 ground-truth entries were
content/docs/references/**pages), used only as evidence for why the denominator must be the tool's declared corpus, not as a live metric.Why (the substance, not just "excludes generated pages")
affected-docs.mjsexcludescontent/docs/references/**by construction — those pages are auto-generated and never hand-edited, so telling an author "you may have affected this page" would be wrong advice, not missing advice. Therefore any recall ratio computed with allcontent/docsedits as the denominator is not measuring this tool at all — it measures how often the docs generator happened to run inside the sampling window (#13306 measured this moving by a factor of 2.7 just from widening the sample window: 17.4% → 6.5%, same tool, same method). The denominator has to be the corpus the tool claims to cover.docs-drift-check.yml's header already records a different invisibility class (a page already in scope that still goes unlisted because it restates a rule without naming what the tool anchors on — #11356/#11357) and already states the check is advisory-only. This PR adds the other half — the denominator statement — and points the two files at each other so they compose instead of each stating their own thing, per the dispatch's Zone 3 suggestion.Fences respected
content/docs/releases/**untouched.Zone 2 (PM mechanism hypotheses) — results
affected-docs.mjsalready had aScope:line (the corpus) but never framed it as a recall denominator, anddocs-drift-check.ymldocumented structural invisibility without ever mentioning a denominator at all. Both are read before writing, and both are now cross-referenced (see the docs-drift-check.yml addition) rather than restated independently in full.affected-docs.mjs --self-test,check-audit-scope.mjs --self-test/run,check-affected-docs.mjs, andcheck-drift-comment.mjsall pin behavior (the corpus enumeration, the anchor derivation, the comment's rendered wording for what the run could/couldn't see) — none of them read or assert on this header comment block. So the denominator is documentation-only right now; it is not mechanically derived from a single source of truth, and this wording is the thing that must not drift.Tests
Comment-only change to two files — no production behavior touched, so no build/test face is owed beyond confirming the edit didn't disturb anything the surrounding self-tests pin. All run in the worktree after
pnpm install:node scripts/check-nul-bytes.mjson both files — clean;grep -naP '[\x00-\x08\x0b\x0c\x0e-\x1f\x7f]'on both — no hits.python3 -c "import yaml; yaml.safe_load(open('.github/workflows/docs-drift-check.yml'))"— parses.node scripts/docs-audit/affected-docs.mjs --self-test→✓ affected-docs self-test: 487 cases pass.node scripts/docs-audit/check-audit-scope.mjs --self-test && node scripts/docs-audit/check-audit-scope.mjs→ both green (189 hand-written docs, 9 release-owned pages).node scripts/docs-audit/check-affected-docs.mjs→ exit 0 (the workflow's own self-test step).node scripts/docs-audit/check-drift-comment.mjs→✓ check-drift-comment: 56 cases pass across 5 fixture diff(s).node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstackagainst the actual 2-file diff (not the dispatch prompt's list) — 29 matched families + 2 convention-triggered ones for editing a gate script. All run, all green (exit codes captured before any pipe):check:agent-test-spelling,check:bash32-floor,check:cli-command-ids,check:cross-package-test-inputs,check:docs-audit-scope,check:entry-guard,check:node-version,check:parse-guard,check:pm-governed-merges,check:pnpm-acquisition,check:pnpm-filter-targets,check:required-contexts,check:shard-attestation,check:stall-guard-budget,check:watch-hint-literal,check:workflow-status-functions,check-aggregator-roster.mjs,check-ci-filter-parity.mjs,check-whole-set-label-write.mjs,pm/ci-failure.mjs --self-test,pm/bare-root-worklist.mjs --self-test,check:pm-dispatch-gates.scripts/docs-audit/**and a workflow header, publishes nothing from any package.skip-changesetlabel applied.PR relationship
Fixes #13306— this PR is the dispatch's own closing condition: "the deliverable is the two headers' wording." #13471 (theSCREAMING_SNAKEliteral-anchor gap) is independent and intentionally not touched here, per the ruling.Generated by Claude Code
Generated by Claude Code