Skip to content

Commit afd593b

Browse files
committed
fix(devx): compare the census+ledger UNION in --fix, so a shifted citation is not a population change
`check-system-context-census --fix` partitioned a file's page anchors into read anchors and ledger-excused citations by LINE: an anchor was a read anchor unless it sat on a line `NON_READ_ANCHORS` currently locates. That compares two different coordinate systems. The page's anchors are pre-shift by construction -- rot is the only reason `--fix` runs -- while a ledger row locates itself by NEEDLE in the current tree. So a file whose ledger-excused citation also moved had that citation counted as a read anchor, and the gate reported a POPULATION change over a population that never moved. Measured twice, in two lanes, on two files: `security-plugin.ts` (7 reads + 1 citation, all displaced +20/+19 by an unrelated bootstrap edit, zero `isSystem` lines added or removed) refused with "page anchors 8 distinct read line(s), census finds 7"; `rest-server.ts` (6 + 2) with "7 ... finds 6", the contradicting `[ledger-row-unused]` line sitting in the same run's output both times. The remedy compares the page's DISTINCT anchor lines against the UNION of the two classes of line the page may anchor -- census read sites and ledger citations -- and aligns them by order. That removes the miscount in both directions, including the dangerous one, and buys a postcondition the per-class comparison cannot state: the rewrite is a bijection onto the file's anchorable lines, so every file `--fix` touches evaluates clean by construction. ⛔ The refusal itself is unchanged as a safety property: a site that genuinely arrives or vanishes is still refused, now with both compared counts, both target classes and the set difference printed, so an author can tell a displacement from an arrival without running the census in two trees by hand. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Pk26oZ12t5N1hwGW1m1MgC
1 parent ae8edd2 commit afd593b

1 file changed

Lines changed: 288 additions & 36 deletions

File tree

scripts/check-system-context-census.mjs

Lines changed: 288 additions & 36 deletions
Original file line numberDiff line numberDiff line change
@@ -100,11 +100,18 @@
100100
*
101101
* ## `--fix` repairs rot and REFUSES to repair population
102102
*
103-
* Per file, when the page's read-anchor count equals the census's site count, the
104-
* two are mapped in line order and the numbers rewritten: that is a pure shift, the
105-
* shape an unrelated edit produces. When the counts differ, the population changed
106-
* -- a site arrived or vanished -- and no mechanical mapping is honest. `--fix`
107-
* leaves those alone and the gate stays red until a human writes the row.
103+
* Per file, when the page's DISTINCT anchor count equals the number of lines the
104+
* file offers to be anchored -- its census read sites AND its `NON_READ_ANCHORS`
105+
* citations, as one union -- the two are mapped in line order and the numbers
106+
* rewritten: that is a pure shift, the shape an unrelated edit produces. When the
107+
* counts differ, the population changed -- a site arrived or vanished -- and no
108+
* mechanical mapping is honest. `--fix` leaves those alone and the gate stays red
109+
* until a human writes the row.
110+
*
111+
* ⭐ The union is load-bearing, not tidiness: subtracting the ledger by LINE
112+
* compares a pre-shift page with a post-shift ledger and reports a POPULATION
113+
* change over a population that never moved (#13490). `fixAnchors` carries the two
114+
* measured occurrences and why the union is the safer shape.
108115
*
109116
* ## Refusals, never quiet passes (#4690)
110117
*
@@ -748,17 +755,103 @@ export function evaluate({
748755
};
749756
}
750757

758+
/** A line list for a refusal message, capped so one bad file cannot flood the log. */
759+
function fmtLines(lines, cap = 14) {
760+
if (lines.length === 0) return '(none)';
761+
const shown = lines.slice(0, cap).join(', ');
762+
return lines.length > cap ? `${shown}, … (+${lines.length - cap} more)` : shown;
763+
}
764+
765+
/**
766+
* The refusal, with everything it compared -- BOTH counts, BOTH target classes,
767+
* and the set difference.
768+
*
769+
* ⭐ Why the sets and not just the counts. Twice now this refusal has been read as
770+
* "your diff added or removed an elevation read site" when nothing of the sort had
771+
* happened, and the output gave the author no way to tell which case they were in
772+
* short of running `isystem-census.mjs --json` in two trees by hand. The last line
773+
* settles it mechanically: if NOTHING is already anchored the page is uniformly
774+
* displaced and some citation is unaccounted for; if everything but one target is
775+
* already anchored, that one target is the site that arrived.
776+
*/
777+
function describeRefusal({ path, pageLines, censusLines, ledgerLines, targets, located }) {
778+
const anchoredSet = new Set(pageLines);
779+
const targetSet = new Set(targets);
780+
const alreadyAnchored = targets.filter((line) => anchoredSet.has(line));
781+
const unanchored = targets.filter((line) => !anchoredSet.has(line));
782+
const stray = pageLines.filter((line) => !targetSet.has(line));
783+
const why = ledgerLines
784+
.map((line) => `${line} (${located.get(`${path}:${line}`)?.why ?? 'declared non-read'})`)
785+
.join('; ');
786+
return (
787+
`${path}: the page anchors ${pageLines.length} distinct line(s) into this file, but the tree ` +
788+
`holds ${targets.length} anchorable line(s) -- ${censusLines.length} census read site(s) plus ` +
789+
`${ledgerLines.length} NON_READ_ANCHORS citation(s). The POPULATION changed, this is not a ` +
790+
'shift. A row has to be written or deleted by hand.\n' +
791+
` page anchors ......... ${fmtLines(pageLines)}\n` +
792+
` census read sites .... ${fmtLines(censusLines)}\n` +
793+
` ledger-excused ....... ${why || '(none)'}\n` +
794+
` already anchored ..... ${alreadyAnchored.length} of ${targets.length} target(s)\n` +
795+
` target, NO anchor .... ${fmtLines(unanchored)}\n` +
796+
` anchor, NO target .... ${fmtLines(stray)}`
797+
);
798+
}
799+
751800
/**
752801
* Rewrite rotted read-site anchors and ledger anchors in place.
753802
*
754-
* Only pure shifts: per file, the page's read-anchor count must equal the census's
755-
* site count. A population change is left for a human.
803+
* Only pure shifts. Per file the page's DISTINCT anchor lines are compared with the
804+
* union of the two classes of line this page is allowed to anchor -- the census's
805+
* read sites and the `NON_READ_ANCHORS` citations -- and rewritten by order when
806+
* the two counts agree. A population change is left for a human.
807+
*
808+
* ## ⛔ Why the ledger cannot be subtracted by LINE (#13490)
809+
*
810+
* The obvious partition -- "a page anchor is a read anchor unless it sits on a
811+
* ledger line" -- compares two DIFFERENT coordinate systems. The page's anchors are
812+
* pre-shift, by construction: rot is the only reason `--fix` is running. The ledger
813+
* lines are post-shift, because a row locates itself by NEEDLE in the current tree.
814+
* So a file whose ledger-excused citation also moved has that citation counted as a
815+
* read anchor, and the gate reports a POPULATION change over a population that
816+
* never moved. Measured twice, in two lanes, on two different files:
817+
*
818+
* security-plugin.ts 7 read sites + 1 ledger citation, all displaced +20/+19 by
819+
* an unrelated bootstrap edit; zero `isSystem` lines added or
820+
* removed. Refusal: "page anchors 8 distinct read line(s),
821+
* census finds 7". (PR #13514, cost a patch round.)
822+
* rest-server.ts 6 read sites + 2 ledger citations, displaced +3/+11 by a
823+
* merge. Refusal: "page anchors 7 ... census finds 6", with
824+
* the contradicting `[ledger-row-unused]` line in the SAME
825+
* run's output.
826+
*
827+
* ⚠️ And it is wrong in the other direction too, which is the dangerous one: a
828+
* stale READ anchor that happens to land on a line the ledger now occupies was
829+
* SUBTRACTED, so the counts could agree by cancellation and the rewrite would map
830+
* the surviving anchors onto each other's rows -- a page that is wrong and GREEN,
831+
* because both classes stay covered. That crossing is real: on the second
832+
* occurrence `rest-server.ts:1267` was simultaneously the second inbound seam's new
833+
* home and a read row's stale anchor.
834+
*
835+
* ⭐ Comparing the UNION removes both directions at once, and buys a postcondition
836+
* the per-class comparison cannot state: the rewrite is a BIJECTION from the page's
837+
* distinct anchor lines onto the file's anchorable lines, so every census site is
838+
* anchored, every ledger row is used and no anchor is unexplained -- for every file
839+
* `--fix` touches, `evaluate` is clean by construction. That is why the union is
840+
* the safer of the two shapes, and it is the one taken: it also refuses when a
841+
* ledger citation was added or dropped without the page following, which comparing
842+
* reads alone would have rewritten straight past.
843+
*
844+
* ⛔ What it still cannot see, stated rather than papered over: alignment is by
845+
* ORDER, so a pure displacement is reconstructed exactly, but a REORDERING that
846+
* moves a cited construct past another one inside the same file is indistinguishable
847+
* from a shift on line numbers alone. No line-only tool can tell those apart -- and
848+
* `evaluate` cannot either, since both classes stay covered. Rows are matched to
849+
* lines by a human there, as they always were.
756850
*
757851
* @returns {{ text: string, rewrites: string[], refused: string[] }}
758852
*/
759853
export function fixAnchors({ pageText, census, tracked, readFile, ledger = NON_READ_ANCHORS }) {
760854
const anchors = extractLineAnchors(pageText);
761-
const sites = siteKeys(census);
762855
const { located } = locateNonReadAnchors(ledger, readFile);
763856
/** ledger target lines, per file */
764857
const ledgerByFile = new Map();
@@ -787,42 +880,26 @@ export function fixAnchors({ pageText, census, tracked, readFile, ledger = NON_R
787880
byFile.get(path).push(anchor);
788881
}
789882
for (const [path, fileAnchors] of byFile) {
790-
const ledgerLines = new Set(ledgerByFile.get(path) ?? []);
791-
const censusLines = [...new Set(census.sites.filter((s) => s.file === path).map((s) => s.line))];
883+
const ledgerLines = [...new Set(ledgerByFile.get(path) ?? [])].sort((a, b) => a - b);
884+
const censusLines = [...new Set(census.sites.filter((s) => s.file === path).map((s) => s.line))].sort(
885+
(a, b) => a - b
886+
);
887+
// ⭐ The comparison is against the UNION of both target classes, in one pass.
792888
// A row cites the same line more than once (`:274` appears in the table AND in
793889
// the rough edges), so the comparable unit is a DISTINCT line, not an anchor.
794-
const readAnchors = fileAnchors.filter((a) => !ledgerLines.has(a.line));
795-
const readLines = [...new Set(readAnchors.map((a) => a.line))].sort((a, b) => a - b);
796-
if (readLines.length !== censusLines.length) {
797-
refused.push(
798-
`${path}: page anchors ${readLines.length} distinct read line(s), census finds ` +
799-
`${censusLines.length} -- the POPULATION changed, this is not a shift. A row has to be ` +
800-
'written or deleted by hand.'
801-
);
890+
const targets = [...new Set([...censusLines, ...ledgerLines])].sort((a, b) => a - b);
891+
const pageLines = [...new Set(fileAnchors.map((a) => a.line))].sort((a, b) => a - b);
892+
if (pageLines.length !== targets.length) {
893+
refused.push(describeRefusal({ path, pageLines, censusLines, ledgerLines, targets, located }));
802894
continue;
803895
}
804-
const target = [...censusLines].sort((a, b) => a - b);
805-
const shift = new Map(readLines.map((line, i) => [line, target[i]]));
806-
for (const anchor of readAnchors) {
896+
const shift = new Map(pageLines.map((line, i) => [line, targets[i]]));
897+
for (const anchor of fileAnchors) {
807898
const to = shift.get(anchor.line);
808899
if (to !== undefined && to !== anchor.line) newLine.set(anchor, to);
809900
}
810901
}
811902

812-
// Ledger anchors: an anchor whose file has exactly one ledger line it is nearest
813-
// to, and which is neither a census site nor already on a ledger line.
814-
for (const [path, fileAnchors] of byFile) {
815-
const ledgerLines = ledgerByFile.get(path) ?? [];
816-
if (ledgerLines.length === 0) continue;
817-
const taken = new Set(fileAnchors.filter((a) => ledgerLines.includes(a.line)).map((a) => a.line));
818-
const free = ledgerLines.filter((l) => !taken.has(l));
819-
const orphans = fileAnchors.filter(
820-
(a) => !ledgerLines.includes(a.line) && !sites.has(`${path}:${a.line}`) && !newLine.has(a)
821-
);
822-
const orphanLines = new Set(orphans.map((a) => a.line));
823-
if (free.length === 1 && orphanLines.size === 1) for (const a of orphans) newLine.set(a, free[0]);
824-
}
825-
826903
// Apply, latest anchor first, so earlier offsets stay valid.
827904
const rewrites = [];
828905
let text = pageText;
@@ -925,12 +1002,67 @@ const FIXTURE_LEDGER = [
9251002
{ file: 'pkg/a.ts', needle: 'export function isSystemObjectName', why: 'name-prefix helper, not a read' },
9261003
];
9271004

1005+
/**
1006+
* ⭐ The CROSSING fixture (#13490). `pkg/a.ts` puts its read ABOVE its ledger
1007+
* citation, which is the easy order: a stale read anchor can never land on the
1008+
* ledger's line. Here the citation sits BELOW the read site, so a displacement
1009+
* walks the citation onto ground a read anchor used to hold -- and the ledger was
1010+
* subtracted by LINE, so that read anchor was subtracted with it. The counts then
1011+
* agreed by cancellation and the rewrite mapped the two surviving anchors onto
1012+
* each other's rows: a page that is WRONG and GREEN, because both classes stay
1013+
* covered and nothing downstream compares a row to its meaning. That crossing is
1014+
* not hypothetical -- `rest-server.ts:1267` was simultaneously the second inbound
1015+
* seam's new home and a read row's stale anchor on the second occurrence.
1016+
*/
1017+
const CROSSING_SOURCE = [
1018+
'export function guard(ctx: ExecutionContext) {', // 1
1019+
' const kind = classify(ctx);', // 2
1020+
' if (isSystemObjectName(ctx.objectName)) return SKIP;', // 3 <- ledger needle
1021+
' audit(kind);', // 4
1022+
' if (ctx.isSystem) return ALLOW;', // 5 <- the elevation read
1023+
' return DENY;', // 6
1024+
'}', // 7
1025+
].join('\n');
1026+
1027+
const CROSSING_CENSUS = {
1028+
...FIXTURE_CENSUS,
1029+
sites: [{ file: 'pkg/b.ts', line: 5, receiver: 'ctx', package: 'pkg', text: 'if (ctx.isSystem) return ALLOW;' }],
1030+
files: ['pkg/b.ts'],
1031+
};
1032+
1033+
const CROSSING_LEDGER = [
1034+
{
1035+
file: 'pkg/b.ts',
1036+
needle: 'isSystemObjectName(ctx.objectName)',
1037+
why: 'the sys_ name-prefix helper call, not a read',
1038+
},
1039+
];
1040+
9281041
function fixtureRead(relPath) {
9291042
if (relPath === 'pkg/a.ts') return FIXTURE_SOURCE;
1043+
if (relPath === 'pkg/b.ts') return CROSSING_SOURCE;
9301044
throw new Error(`no fixture for ${relPath}`);
9311045
}
9321046

933-
const FIXTURE_TRACKED = ['pkg/a.ts', 'other/a.ts'];
1047+
const FIXTURE_TRACKED = ['pkg/a.ts', 'other/a.ts', 'pkg/b.ts'];
1048+
1049+
/**
1050+
* One anchor per line, so the two slots can be told apart AFTER a rewrite: the
1051+
* question these cases ask is not "are both lines covered" -- the buggy fixer
1052+
* covered both -- but "did each ROW keep its own line".
1053+
*/
1054+
function crossingPage({ read = 'pkg/b.ts:5', helper = 'pkg/b.ts:3' } = {}) {
1055+
return [
1056+
'---',
1057+
'title: crossing fixture',
1058+
'---',
1059+
'',
1060+
'the elevation read at `' + read + '`.',
1061+
'',
1062+
'the name helper at `' + helper + '`.',
1063+
'',
1064+
].join('\n');
1065+
}
9341066

9351067
/** A one-row stand-in for `DECLARED_COUNTS`, so the fixtures need one sentence. */
9361068
const FIXTURE_COUNTS = [
@@ -1249,6 +1381,126 @@ function selfTest() {
12491381
JSON.stringify(refusedFix.refused)
12501382
);
12511383

1384+
// ── ⭐ #13490: the incident shape -- reads AND ledger citations BOTH shift ────
1385+
//
1386+
// The pre-existing case above shifts the read only, which is why it never caught
1387+
// this: a page anchor is pre-shift by construction, a ledger line is located by
1388+
// needle in the CURRENT tree, and subtracting one from the other counts the
1389+
// displaced citation as a read anchor. Measured on two files in two lanes --
1390+
// `security-plugin.ts` (7 reads + 1 citation, all +20/+19, zero `isSystem` lines
1391+
// added or removed) refused with "page anchors 8 distinct read line(s), census
1392+
// finds 7", and `rest-server.ts` (6 + 2) with "7 ... finds 6".
1393+
const bothShifted = fixAnchors({
1394+
pageText: fixturePage({ anchor: 'pkg/a.ts:1', helper: 'pkg/a.ts:6' }),
1395+
census: FIXTURE_CENSUS,
1396+
tracked: FIXTURE_TRACKED,
1397+
readFile: fixtureRead,
1398+
ledger: FIXTURE_LEDGER,
1399+
});
1400+
t(
1401+
'FIX #13490: a shift that moves the LEDGER citation too is a shift, not a population change',
1402+
bothShifted.refused.length === 0 &&
1403+
bothShifted.text.includes('`pkg/a.ts:2`') &&
1404+
bothShifted.text.includes('`pkg/a.ts:7`'),
1405+
`refused=${JSON.stringify(bothShifted.refused)} rewrites=${JSON.stringify(bothShifted.rewrites)}`
1406+
);
1407+
1408+
// ⭐ The postcondition the union buys: the rewrite is a BIJECTION from the page's
1409+
// distinct anchor lines onto the file's anchorable lines, so a file `--fix`
1410+
// touched cannot come back with a missing site, an unexplained anchor or an
1411+
// unused ledger row. Pinned behaviourally rather than argued in a comment.
1412+
const afterFix = evaluate({
1413+
pageText: bothShifted.text,
1414+
census: FIXTURE_CENSUS,
1415+
tracked: FIXTURE_TRACKED,
1416+
readFile: fixtureRead,
1417+
ledger: FIXTURE_LEDGER,
1418+
declaredCounts: [],
1419+
unenforcedCounts: [],
1420+
});
1421+
t(
1422+
'FIX #13490: what --fix rewrote evaluates clean -- every site anchored, every ledger row used',
1423+
afterFix.problems.length === 0,
1424+
afterFix.problems.join(' | ')
1425+
);
1426+
1427+
// ── ⛔ the dangerous direction: the citation crosses onto a read anchor's line ─
1428+
//
1429+
// Subtracting the ledger by LINE removed the stale READ anchor here (it sits on
1430+
// `:3`, the citation's new home), the counts agreed by cancellation, and the one
1431+
// surviving anchor was mapped onto the read site -- leaving the page GREEN with
1432+
// the two rows pointing at each other's lines. Both spellings survive either
1433+
// way, so this case asserts which ROW holds which line.
1434+
const crossed = fixAnchors({
1435+
pageText: crossingPage({ read: 'pkg/b.ts:3', helper: 'pkg/b.ts:2' }),
1436+
census: CROSSING_CENSUS,
1437+
tracked: FIXTURE_TRACKED,
1438+
readFile: fixtureRead,
1439+
ledger: CROSSING_LEDGER,
1440+
});
1441+
t(
1442+
'FIX #13490: a citation crossing a read anchor keeps each ROW on its own line, not merely covered',
1443+
crossed.refused.length === 0 &&
1444+
crossed.text.includes('the elevation read at `pkg/b.ts:5`') &&
1445+
crossed.text.includes('the name helper at `pkg/b.ts:3`'),
1446+
`refused=${JSON.stringify(crossed.refused)} rewrites=${JSON.stringify(crossed.rewrites)}`
1447+
);
1448+
1449+
// ── ⭐ and the safety property, on the shape that now ACCEPTS ────────────────
1450+
//
1451+
// ⛔ The fix must not buy acceptance with the refusal. A read site ARRIVES while
1452+
// the ledger citation shifts: the old counting arm and the new one both refuse
1453+
// here, and that must stay true, or #13490 was closed by deleting the guard.
1454+
const grewWhileShifting = fixAnchors({
1455+
pageText: fixturePage({ anchor: 'pkg/a.ts:2', helper: 'pkg/a.ts:6' }),
1456+
census: arrived,
1457+
tracked: FIXTURE_TRACKED,
1458+
readFile: fixtureRead,
1459+
ledger: FIXTURE_LEDGER,
1460+
});
1461+
t(
1462+
'FIX #13490: a site that ARRIVES while the citation shifts is still REFUSED',
1463+
grewWhileShifting.rewrites.length === 0 && grewWhileShifting.refused.length === 1,
1464+
JSON.stringify(grewWhileShifting.refused)
1465+
);
1466+
1467+
const vanished = fixAnchors({
1468+
pageText: crossingPage(),
1469+
census: { ...CROSSING_CENSUS, sites: [...CROSSING_CENSUS.sites, { file: 'pkg/b.ts', line: 6, receiver: 'ctx', package: 'pkg', text: 'return DENY;' }] },
1470+
tracked: FIXTURE_TRACKED,
1471+
readFile: fixtureRead,
1472+
ledger: CROSSING_LEDGER,
1473+
});
1474+
t(
1475+
'FIX #13490: an unanchored site in the crossing file is REFUSED too',
1476+
vanished.rewrites.length === 0 && vanished.refused.length === 1,
1477+
JSON.stringify(vanished.refused)
1478+
);
1479+
1480+
// ── the refusal has to SHOW its work (both counts, both classes, the diff) ────
1481+
//
1482+
// Twice this refusal was read as "your diff added or removed an elevation read
1483+
// site" when nothing had, and the output gave no way to tell which case you were
1484+
// in short of running the census in two trees by hand. `already anchored 8 of 9`
1485+
// + one named target settles it; `0 of 9` says uniform displacement.
1486+
const refusalText = grewWhileShifting.refused[0] ?? '';
1487+
t(
1488+
'FIX #13490: the refusal states BOTH counts it compared and the ledger it set aside',
1489+
refusalText.includes('the page anchors 2 distinct line(s)') &&
1490+
refusalText.includes('holds 3 anchorable line(s)') &&
1491+
refusalText.includes('2 census read site(s)') &&
1492+
refusalText.includes('1 NON_READ_ANCHORS citation(s)') &&
1493+
refusalText.includes('name-prefix helper, not a read'),
1494+
refusalText
1495+
);
1496+
t(
1497+
'FIX #13490: the refusal names the set difference, not just a count',
1498+
refusalText.includes('already anchored') &&
1499+
refusalText.includes('target, NO anchor') &&
1500+
refusalText.includes('anchor, NO target'),
1501+
refusalText
1502+
);
1503+
12521504
process.stdout.write(
12531505
failures === 0
12541506
? '\ncheck-system-context-census --self-test: all cases passed\n'

0 commit comments

Comments
 (0)