Skip to content

feat(CC-514): cross-document drift lint for GENERATED policy tables - #545

Merged
screenleon merged 2 commits into
mainfrom
feat/CC-514
Aug 27, 2026
Merged

feat(CC-514): cross-document drift lint for GENERATED policy tables#545
screenleon merged 2 commits into
mainfrom
feat/CC-514

Conversation

@screenleon

Copy link
Copy Markdown
Owner

Summary

  • Delivers CC-514's remaining Req 5 (cross-document lint) and Req 6 (drift ratchet), collapsed into one mechanism per pre-impl design review: a new tools/lint/check-policy-doc-sync.sh dynamically discovers every <!-- BEGIN GENERATED: <source> --> markdown table in any doc and verifies it still matches the core/policy/* file it claims to mirror. Dynamic discovery is itself the ratchet — a new marked block anywhere is checked the moment it exists, with no separate registration step, so Req 6 doesn't need a second artifact.
  • Five TSV-sourced blocks use a generic comparator (render the source as markdown rows, compare whole lines — cell-splitting would misparse the policy-signals table's regex patterns, which contain unescaped literal |). The one YAML-sourced block (core/policy/reviewer-policy.yaml) gets a dedicated comparator rather than a general YAML-to-table renderer — a scoped tradeoff confirmed with the user, revisited only if a second YAML-shaped source actually shows up.
  • Wired into .github/workflows/lint.yml (a direct real-repo run plus its own regression-test job) and the tests/lib/test-suite-runner.sh / tests/shell/test-run-all-tests.sh suite registries.
  • Updated docs/delivery-assurance-map.md's own provenance note and "no automated lint" disclaimer, since both had gone stale as of this PR. CC-514 fully closes (all 7 requirements now delivered).

Review history

  • Mandatory reuse/simplify pass (4 parallel review angles) found and fixed 4 real issues before the first gate round: the YAML comparator read its source twice and cell-split doc rows instead of the whole-line strategy its TSV sibling used (rewritten to one pass, one strategy); the doc-file marker scan forked awk twice per file for overlapping work (merged into one pass); git ls-files without --others silently skipped a brand-new, not-yet-git added doc file, contradicting the script's own "no registration step" claim (fixed, with a new regression case).
  • pr-gate round 1 (codex, parallel, standard tier): NO-GO — 3 real findings, all fixed:
    • qa-tester: doc-side extraction silently discarded any non-table-row line inside a block, so injected prose/markup would go unnoticed. Fixed: any non-blank, non-table-row line inside a block is now rejected outright.
    • security-reviewer: a marker's source path is attacker-controlled (PR-authored doc text) and was read with no containment check — a symlinked or path-traversing source could disclose arbitrary file content in CI logs. Fixed: rejects a symlinked source outright and verifies the fully resolved path lands exactly where the literal, unresolved concatenation says it should, before reading anything.
    • risk-reviewer: a nested BEGIN marker silently overwrote the outer block's own bookkeeping, leaving it completely unverified — a false pass. Fixed: nested markers are now a distinct, explicit rejection.
  • pr-gate round 2 (targeted, sequential): GO — all five reviewers approved.

Test plan

  • bash tests/shell/test-check-policy-doc-sync.sh (15/15, run 3x for stability)
  • bash tools/lint/check-policy-doc-sync.sh clean against the real repo (6 blocks across 1 file)
  • tools/lint/lint-shellcheck.sh clean on the whole repo
  • tools/lint/lint-test-suite-registry.sh — registry/CI parity holds
  • pmctl gate run --executor codex --policy generic → GO (2 rounds; round 1's 3 real findings fixed)
  • tests/bin/run-all-tests.sh → 105 passed, 0 failed, 0 skipped

🤖 Generated with Claude Code

https://claude.ai/code/session_0177ds9nxEMtsh3bwpBxNHYx

screenleon and others added 2 commits August 27, 2026 03:13
Delivers CC-514's remaining Req 5 (cross-document lint) and Req 6 (drift
ratchet), collapsed into one mechanism per pre-impl design review: a new
tools/lint/check-policy-doc-sync.sh dynamically discovers every
"<!-- BEGIN GENERATED: <source> -->" markdown table in any doc (tracked or
not-yet-staged) and verifies it still matches the core/policy/* file it
claims to mirror. Dynamic discovery is itself the ratchet -- a new marked
block anywhere is checked the moment it exists, with no separate
registration step, so Req 6 doesn't need a second artifact.

Five TSV-sourced blocks use a generic comparator (render the source as
markdown rows, compare whole lines -- cell-splitting would misparse the
policy-signals table's regex patterns, which contain unescaped literal
"|"). The one YAML-sourced block (core/policy/reviewer-policy.yaml) gets a
dedicated comparator rather than a general YAML-to-table renderer -- a
scoped tradeoff confirmed with the user, revisited only if a second
YAML-shaped source actually shows up.

Wired into .github/workflows/lint.yml (a direct real-repo run plus its own
regression-test job) and the tests/lib/test-suite-runner.sh /
tests/shell/test-run-all-tests.sh suite registries.

The mandatory reuse/simplify pass (4 parallel review angles) found and
fixed four real issues before this landed: the YAML comparator read its
source twice and cell-split doc rows instead of the whole-line strategy
its TSV sibling already used (rewritten to one pass, one strategy, cutting
~40 lines); the doc-file marker scan forked awk twice per file for
overlapping work (merged into one pass); and git ls-files without
--others silently skipped a brand-new, not-yet-`git add`ed doc file,
contradicting the script's own "no registration step" claim (fixed, with
a new regression case for both that gap and a stray END marker).

Also updated docs/delivery-assurance-map.md's own provenance note and its
"no automated lint" disclaimer, since both had gone stale as of this PR.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0177ds9nxEMtsh3bwpBxNHYx
Three real findings from gate round 1, all in the same script:

- qa-tester: the doc-side extraction filtered for "|"-prefixed lines and
  silently discarded anything else, so injected prose or markup inside an
  otherwise-correct GENERATED block would pass unnoticed. Now any non-blank,
  non-table-row line inside a block is rejected outright.
- security-reviewer: a marker's source path is attacker-controlled (it's
  whatever text a PR writes inside a doc), and the checker read it with no
  containment check -- a symlinked or path-traversing source could disclose
  arbitrary file content in CI logs via the diff output. Now rejects a
  symlinked source outright and verifies the fully resolved path lands
  exactly where the literal, unresolved concatenation says it should,
  before reading anything.
- risk-reviewer: a BEGIN encountered while a block was already open
  silently overwrote that block's own source/start bookkeeping, so a
  nested marker pair left the outer block completely unverified -- a false
  pass that defeats the whole drift-detection property. Now rejected as a
  distinct, explicit "nested markers are not supported" error.

Added one regression case per finding, plus adjusted the pre-existing
unmatched-marker test (which happened to also trigger the new nested-marker
path with its original mutation shape) to isolate the two failure modes.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0177ds9nxEMtsh3bwpBxNHYx
@screenleon
screenleon merged commit 6169849 into main Aug 27, 2026
66 checks passed
@screenleon
screenleon deleted the feat/CC-514 branch August 27, 2026 03:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant