feat(CC-514): cross-document drift lint for GENERATED policy tables - #545
Merged
Conversation
Delivers CC-514's remaining Req 5 (cross-document lint) and Req 6 (drift ratchet), collapsed into one mechanism per pre-impl design review: a new tools/lint/check-policy-doc-sync.sh dynamically discovers every "<!-- BEGIN GENERATED: <source> -->" markdown table in any doc (tracked or not-yet-staged) and verifies it still matches the core/policy/* file it claims to mirror. Dynamic discovery is itself the ratchet -- a new marked block anywhere is checked the moment it exists, with no separate registration step, so Req 6 doesn't need a second artifact. Five TSV-sourced blocks use a generic comparator (render the source as markdown rows, compare whole lines -- cell-splitting would misparse the policy-signals table's regex patterns, which contain unescaped literal "|"). The one YAML-sourced block (core/policy/reviewer-policy.yaml) gets a dedicated comparator rather than a general YAML-to-table renderer -- a scoped tradeoff confirmed with the user, revisited only if a second YAML-shaped source actually shows up. Wired into .github/workflows/lint.yml (a direct real-repo run plus its own regression-test job) and the tests/lib/test-suite-runner.sh / tests/shell/test-run-all-tests.sh suite registries. The mandatory reuse/simplify pass (4 parallel review angles) found and fixed four real issues before this landed: the YAML comparator read its source twice and cell-split doc rows instead of the whole-line strategy its TSV sibling already used (rewritten to one pass, one strategy, cutting ~40 lines); the doc-file marker scan forked awk twice per file for overlapping work (merged into one pass); and git ls-files without --others silently skipped a brand-new, not-yet-`git add`ed doc file, contradicting the script's own "no registration step" claim (fixed, with a new regression case for both that gap and a stray END marker). Also updated docs/delivery-assurance-map.md's own provenance note and its "no automated lint" disclaimer, since both had gone stale as of this PR. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0177ds9nxEMtsh3bwpBxNHYx
Three real findings from gate round 1, all in the same script: - qa-tester: the doc-side extraction filtered for "|"-prefixed lines and silently discarded anything else, so injected prose or markup inside an otherwise-correct GENERATED block would pass unnoticed. Now any non-blank, non-table-row line inside a block is rejected outright. - security-reviewer: a marker's source path is attacker-controlled (it's whatever text a PR writes inside a doc), and the checker read it with no containment check -- a symlinked or path-traversing source could disclose arbitrary file content in CI logs via the diff output. Now rejects a symlinked source outright and verifies the fully resolved path lands exactly where the literal, unresolved concatenation says it should, before reading anything. - risk-reviewer: a BEGIN encountered while a block was already open silently overwrote that block's own source/start bookkeeping, so a nested marker pair left the outer block completely unverified -- a false pass that defeats the whole drift-detection property. Now rejected as a distinct, explicit "nested markers are not supported" error. Added one regression case per finding, plus adjusted the pre-existing unmatched-marker test (which happened to also trigger the new nested-marker path with its original mutation shape) to isolate the two failure modes. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0177ds9nxEMtsh3bwpBxNHYx
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
tools/lint/check-policy-doc-sync.shdynamically discovers every<!-- BEGIN GENERATED: <source> -->markdown table in any doc and verifies it still matches thecore/policy/*file it claims to mirror. Dynamic discovery is itself the ratchet — a new marked block anywhere is checked the moment it exists, with no separate registration step, so Req 6 doesn't need a second artifact.|). The one YAML-sourced block (core/policy/reviewer-policy.yaml) gets a dedicated comparator rather than a general YAML-to-table renderer — a scoped tradeoff confirmed with the user, revisited only if a second YAML-shaped source actually shows up..github/workflows/lint.yml(a direct real-repo run plus its own regression-test job) and thetests/lib/test-suite-runner.sh/tests/shell/test-run-all-tests.shsuite registries.docs/delivery-assurance-map.md's own provenance note and "no automated lint" disclaimer, since both had gone stale as of this PR. CC-514 fully closes (all 7 requirements now delivered).Review history
awktwice per file for overlapping work (merged into one pass);git ls-fileswithout--otherssilently skipped a brand-new, not-yet-git added doc file, contradicting the script's own "no registration step" claim (fixed, with a new regression case).pr-gateround 1 (codex, parallel, standard tier): NO-GO — 3 real findings, all fixed:pr-gateround 2 (targeted, sequential): GO — all five reviewers approved.Test plan
bash tests/shell/test-check-policy-doc-sync.sh(15/15, run 3x for stability)bash tools/lint/check-policy-doc-sync.shclean against the real repo (6 blocks across 1 file)tools/lint/lint-shellcheck.shclean on the whole repotools/lint/lint-test-suite-registry.sh— registry/CI parity holdspmctl gate run --executor codex --policy generic→ GO (2 rounds; round 1's 3 real findings fixed)tests/bin/run-all-tests.sh→ 105 passed, 0 failed, 0 skipped🤖 Generated with Claude Code
https://claude.ai/code/session_0177ds9nxEMtsh3bwpBxNHYx