Content-type-aware diff: tables (CSV/Excel) and PDF

[gregoryfoster]

Scope

Content-type-aware normalizers and renderers for the diff surface:

1. Tables — CSV, Excel
2. PDF — page-aware

Dependencies rewritten 2026-08-19 in a backlog staleness pass. The design below is unaffected by the service-boundary rework and still wanted; the blocker chain in the original was stale in both directions. See Dependencies — this is now build-from-scratch behind #222, not an extension of an existing module.

Tables

• Row-key-aware normalization: when the extractor identifies a key column, diff by row-key (row add/remove/modified) instead of line-by-line.
• Render in dashboard as a data-table with adds highlighted green, removes red, per-cell changes yellow with old → new values visible.
• Requires: pandas or equivalent; extractor metadata to carry key-column hints.

PDF

• Page-aware diff: show which pages changed, diff extracted text per-page rather than as one flat blob.
• Integrate with page-level screenshot comparison where available.

Dependencies

Blocked by #222, not #115.

The original read "Blocked by #115 Phases A/B (needs src/core/diff/ module, normalization layer, unified-diff core)". That is stale twice over:

• Watch Change UI: GitHub-style diff viewer (Phase A + B.1 done; B.2/C/D descoped) #115 is closed (2026-04-28 — "Phase A + B.1 done; B.2/C/D descoped").
• The module it delivered was then deleted. There is no src/core/diff/. Phase 5 (Phase 5 — refactor Watcher to produce SourceRevisions in Archiver (v2 cutover) #156) dropped the Snapshot/Change tables and the diff engine; Notification Template UI/UX polish: event labels, Context layout, and strip the dead diff/significance surface #221 stripped the remaining inert surface. compute_unified_diff is gone from src/ entirely.

So there is no diff layer to extend. The real chain is:

#117 (this)  →  #222 (restore unified diff + chunk semantics)  →  a persistent content store

ChangeRevision stores a content fingerprint and nothing else, so at dispatch time there is no previous content to diff against — for any content type. Watcher is currently a fingerprint-shift notifier: it reports that an item changed, not what changed. Content-type-aware diffing is a refinement of a capability that does not presently exist, so this cannot start until #222 lands.

Consequence for scoping: the acceptance criterion "src/core/diff/normalize.py gains normalize_table and normalize_pdf_text" presumes a file that must first be recreated. Whoever does #222 should decide whether to reintroduce src/core/diff/ as a package — if they do, this issue extends it as originally imagined; if they restore only the notification-side helpers from the 89d64eb anchor named in #222, this issue also owns creating the module.

Legacy .xls — updated

The 2026-06-23 note from #168 still holds, with one change in cost:

• CsvExcelExtractor is backed by openpyxl, which reads .xlsx (OOXML) only — it cannot parse the old binary .xls.
• application/vnd.ms-excel is therefore absent from _DEFAULT_EXTRACTOR_MAP (src/core/registry.py) and .xls is absent from the URL-extension tiebreaker (src/core/media_type.py); both resolve to the HTML fallback rather than mis-parsing. Verified still true.
• What changed: extraction moved to co-core — the map now imports co_core.pure.extract.csv_excel.CsvExcelExtractor. Adding a legacy .xls reader (xlrd, or an upstream convert-to-.xlsx step) is now a cannobserv change, not a local one, and per the cross-repo policy needs an issue filed there rather than an edit here. That raises its cost enough that it should be an explicit decision, not a silent prerequisite.

.xlsx, .csv, and .pdf dispatch correctly.

Acceptance

• Table normalizer (row-key-aware) and PDF normalizer (page-aware), in whatever module Restore change detail (unified diff + chunk semantics) once Replicator provides a persistent content store #222 establishes
• Dashboard Change Detail supports content-type-aware rendering for CSV/Excel and PDF items
• TDD coverage for new normalizers
• Legacy .xls: either an upstream cannobserv issue filed, or explicitly declared out of scope

[gregoryfoster]

Note from #168 — legacy .xls is currently unsupported

#168 (slice 2) revived content-type→extractor dispatch. Worth flagging for the table-diff work here: legacy .xls (application/vnd.ms-excel) is not routed to the CSV/Excel extractor and falls back to HTML.

• CsvExcelExtractor is backed by openpyxl, which reads .xlsx (OOXML) only — it cannot parse the old binary .xls format.
• So application/vnd.ms-excel is deliberately absent from _DEFAULT_EXTRACTOR_MAP and .xls is absent from the URL-extension tiebreaker (src/core/media_type.py); both resolve to the HTML fallback rather than mis-parsing.

When this issue builds content-type-aware table diffs, .xls needs a separate reader (e.g. xlrd for legacy binaries, or an upstream convert-to-.xlsx step) before it can route to a tabular extractor. .xlsx, .csv, and .pdf already dispatch correctly.