docs(ssot): land the untracked v1.5 authority, make the ledger keep its promises, resolve ten doc-vs-code contradictions - #324
Open
Prekzursil wants to merge 4 commits into
Open
docs(ssot): land the untracked v1.5 authority, make the ledger keep its promises, resolve ten doc-vs-code contradictions#324Prekzursil wants to merge 4 commits into
Prekzursil wants to merge 4 commits into
Conversation
![sentry[bot]](https://avatars.githubusercontent.com/in/12637?s=60&v=4){kind=small}
added 4 commits
August 1, 2026 04:58
…ing config The v1.5 planning corpus — 14 documents, ~440 KB of prose plus 2.5 MB of shell-audit screenshots — existed only in `~/.reframe-review/` on one machine. It was never reviewable, never diffable, and one disk failure from gone. This lands it verbatim, so the corrections in the follow-up passes arrive as reviewable diffs rather than pre-applied. .gitignore, two narrowings `.claude/` -> `.claude/*` + negations. A trailing-slash directory pattern makes every `!` inert, because git cannot re-include a file whose parent directory is excluded. Measured: a brand-new `.claude/commands/x.md` resolved to `.gitignore:63` and was silently dropped, while six already-tracked files kept working (gitignore does not affect tracked files). That split is invisible until someone adds a command and it never lands. Machine-local state stays ignored. `.audit/` stays FULLY ignored — deliberately not narrowed. Its scripts did need to escape a directory declared disposable (`git clean -xfd` was destroying the regeneration recipe for the tracked ledger, i.e. the very property that made deleting the derived bytes safe). But un-ignoring them in place would have tracked each in two locations, which is the duplication this pass exists to remove. They MOVED instead, to `docs/validation/tools/`. Verified both directions, 14 probes: every path that must become trackable does, every one of the ~10 MB of derived bytes stays ignored, and no already-tracked file becomes ignored (which would be a silent-delete risk). The audit ledger now keeps its own promises `v15-audit-ledger.md:27` advertised `| REFUTED | ... | 94 |` while containing zero of them, so the one thing that row exists for — stopping a refuted finding from being re-raised — did not work. All 94 are appended, parsed and count-checked against the advertised 4-critical / 90-high split. `:57-59` pointed at "the untracked full ledger". That is now `docs/validation/v15-audit-ledger-unverified.md.gz` — tracked, 6.66 MB -> 2.26 MB, gzip round-trip verified. It is the sole copy of 2348 findings and no longer lives on one machine. `:20` pointed at `.audit/join_by_agentid.py` "(untracked)". Now tracked. The "3 failing e2e-sidecar tests ... separate, pre-existing defect" note is CLOSED and annotated with the actual cause, which was not a product defect: a test fixture set only the legacy `routing.perFunction` and never `routingPolicy`, and the M3 policy is deliberately authoritative over it AND fail-closed. Plan corrections found while executing it The migration plan claimed `.audit/AUDIT-LEDGER-TIERED.md` has "zero unique content" and is delete-safe. Measured against RECOVERED (the file the claim is actually about — comparing it to the TRACKED ledger, as a first pass did, measures a pair nobody claimed anything about): 42/50 blocks present, 36 byte-identical, and 8 absent from BOTH. It is NOT delete-safe and stays untracked, undeleted. `AUDIT-LEDGER-FINAL.md` IS a duplicate as claimed — 131/131 present, and the one "differing" block is an artifact of the REFUTED append above. Neither file is archived into the repo: with its unique content extracted, archiving 1.17 MB of verified duplicate would recreate the second source of truth.
…t-based
Two defects in the tool this pass just promoted to docs/validation/tools/.
1. It hardcoded `ROOT = Path(__file__).resolve().parent.parent`, which was correct
at its authoring location (`.audit/`) and resolves to `docs/validation/` after the
move. Every relative read then missed and it reported 26 of 36 claims as
mismatched — a detector failing loudly in the wrong direction, blaming the repo
for its own relocation. It now walks up to the markers that exist only at the root
(`.git` + `app/package.json`). Verified from three different cwds (repo root,
`app/`, `$HOME`): all three now agree, 40/40.
2. Roughly half its claims asserted the BROKEN state — e.g. "ai-program/PLAN.md still
denies .coverage-thresholds.json exists" expected True. As a one-shot pre-flight
that was right; as a committed drift detector it inverts the meaning of green: a
passing run would have meant "still broken", and fixing a doc would have turned the
gate red. Claims are now split:
[INV ] 22 invariants — these gate the exit code
[OPEN] 18 known-stale docs later phases will correct — reported, never fail,
and flagged `NOW-FIXED (retire it from OPEN_ITEMS)` when one flips
Five new invariants cover what this pass actually fixed, so a regression is caught
rather than re-discovered:
C12b the ledger's REFUTED section exists AND holds exactly 94 entries with the
4-critical / 90-high split its own summary row at :27 advertises
C12c the unverified ledger is tracked and its gzip actually decompresses (a
corrupt blob would be a silent sole-copy loss)
C12d the ledger no longer points at an "untracked full ledger"
C12e all 3 recovery tools live outside the disposable .audit/ tree
C0.2 a brand-new `.claude/commands/*.md` is trackable, i.e. the gitignore
narrowing has not regressed
basedpyright does not scan this path (`include` is sidecar/media_studio +
sidecar/contract), confirmed by running the pinned 1.39.8 as CI does: 0 errors.
…code's favour Each of these was a document asserting something the code had already changed. None was the reverse, which is why the anti-drift rule is "when a doc and the code disagree, the code wins and the doc is a bug". SpeechBrain — a file contradicting itself, resolved AGAINST the prose `sidecar/pyproject.toml` said "PINNED to 1.0.3 ... 1.1.0 is DELIBERATELY AVOIDED" six lines above a line reading `"speechbrain==1.1.0"`, and three docs repeated the 1.0.3 value. The migration plan called this BLOCKING and expected it to need a live install to settle. It did not. Two independent code sites decide it: `diarize_backend.py` probes for 1.1.0's `utils.fetching.FetchConfig`, and `pinned_fetch_kwargs`'s own docstring says "sidecar/pyproject.toml pins speechbrain==1.1.0, so emitting the 1.0.x shape unconditionally would break the diarizer outright". 1.1.0's shape also pins the checkpoint TENSORS, not just hyperparams.yaml/custom.py — strictly wider. The pin is correct and deliberate; the comment and the three docs are the stale half. The avoidance rationale is also not reachable. Measured against both real wheels: `integrations/k2_fsa` is ABSENT in 1.0.3 and PRESENT in 1.1.0 (8 modules), where its `__init__` re-raises ImportError without `k2` — but 1.1.0 reaches subpackages through `lazy_export_all`, so it is only touched on explicit attribute access, and `_import_speechbrain()` imports only `speechbrain.inference`'s VAD/EncoderClassifier. Chatterbox torch — a silent 7.5 GB re-download, now guarded by a test `requirements-chatterbox.txt` said `torch==2.11.0+cu128`; the CHATTERBOX_REQUIREMENTS tuple says `2.10.0+cu128`. `assets/manager.py:617` decides "installed" by comparing the sentinel's recorded list against `entry.requirements` (the TUPLE), so a divergence means a correctly-built env never counts as installed and re-downloads ~7.5 GB. History: the tuple's 2.10.0 is a deliberate titled decision (#211, "modernize chatterbox voice-clone env to py3.14 + torch 2.10.0 (cu128)"); the .txt's 2.11.0 arrived via the #259 dependabot bump and was "re-paired" in place by #260 without following the tuple. Both trios are real and paired on the cu128 index for py3.14 (verified), so nothing failed loudly — the only symptom was the re-download. The .txt (which calls itself the mirror) now follows the tuple, AND a new test compares the two lists directly. Two sibling tests asserted the .txt's literals and the tuple's order but nothing compared them, which is why the desync was invisible; the order assertions now derive from the tuple instead of restating the version a third time. Both-states verified: re-introducing the desync turns the new test red. The rest * QUALITY-CHARTER declared `oxlint 1.70.0`; `.pre-commit-config.yaml` runs 1.69.0 and is the only oxlint version in the repo. The config's excuse ("npm pins 1.70.0") was false — oxlint is in no tracked package.json. Both now read 1.69.0. * CONTRACTS.md asserted "no keystore, no egress-consent framework" / "no keystore, no consent framework" while `app/main/keystore.ts`, `models/consent.py`, `models/spend_ledger.py` and `components/ConsentToggle.tsx` all ship. * `docs/rpc-contract-v2.md` still said "Not merged; not yet load-bearing". It merged in #282 and IS load-bearing for the coverage gate. * Three plan docs told readers `.coverage-thresholds.json` "does NOT exist — do not reference it". It was added in e38d5d3 (#235), after all three. * `catalog.py` and `test_catalog.py` cited `docs/providers/CATALOG-SEED.md`, a path that has never existed; the file is `docs/plans/provider-hub/CATALOG-SEED.md`. * README claimed "five top-level sections" and four Settings sub-tabs; App.tsx has eight rails and Settings.tsx eight panels. Both lists are now derived from the code. * README named `media-studio-1.4.1-win-x64.*` while package.json reads 1.4.2 — and naming ANY version was wrong, because the newest published release is v1.4.0 (v1.4.1 is a Draft, v1.4.2 untagged), so it pointed at an asset nobody could download. Now version-agnostic, as electron-builder derives the name anyway. * CHANGELOG had no `[1.4.2]` section while README:151 sends readers there for release history. Added, marked unreleased, derived from the 70 commits since v1.4.1. Declined one plan item: it claims `app/main/main.ts` and `security.ts` cite the dead CONTRACTS clause. They cite §0/§7 for "no auth, no network servers, no telemetry" — all still true. Nothing to correct. The verifier now guards all of it: 7 items moved from OPEN to INVARIANT (29 invariants, 11 open), so any of these corrections regressing turns it red instead of being re-discovered.
Dispatch 30677298418 confirmed the two earlier e2e fixes worked — golden-journey now passes on Windows, e2e-sidecar is green, and the VISUAL baseline REGEN step ran for the first time. It also surfaced three NEW failures in preview.spec.ts, identical on windows-latest AND macos-latest, so not platform flake: x 16 preview.spec.ts:151 Advanced disclosure actually COLLAPSES the Deliver cluster x 17 preview.spec.ts:211 Workspace tabs mount, including SemanticSearch x 18 preview.spec.ts:238 export action yields a real file Three failures, ONE cause, plus a second defect that turned one into three. 1. Unpinned viewport. This spec makes a GEOMETRY assertion — `toBeInViewport()` on the Advanced toggle — and never pinned the renderer viewport, so it inherited whatever the CI display handed the BrowserWindow. It came back "viewport ratio 0": entirely off-screen. The assertion was written against a LOCAL measurement (innerWidth 1264, toggle at x 1175..1268), which does not hold on a CI runner. `main.ts` creates the window at 1280x820 and the visual suite already pins exactly that, for exactly this reason — "Pin the viewport so layout (and thus pixels) is reproducible run-to-run" (`_visualSetup.ts`). This reuses its exported WINDOW_WIDTH / WINDOW_HEIGHT rather than restating the numbers a third time. 2. State leaked on failure. `playwright.config.ts` runs `fullyParallel: false, workers: 1` against ONE app, so all tests share live renderer state. The disclosure test restored `advancedOpen = false` at the END of its body — which does not run when an assertion earlier in the body fails. So the single geometry failure left the cluster OPEN and took the next two tests down with it. An `afterEach` now collapses it unconditionally, idempotently, and swallows its own errors so a teardown problem can never mask the real failure. The in-body restore stays: it ASSERTS collapsing works, where the afterEach only GUARANTEES the state. Deleting either loses something, so both carry a note saying so. Typechecked by the `gate-types tsc e2e harness` step added earlier — the first change to `app/e2e/**` that any blocking gate has actually read.
Prekzursil
force-pushed
the
docs/ssot-phase0-1
branch
from
August 1, 2026 01:58
28c41d9 to
7e19c0d
Compare
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 28c41d9531
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
Comment on lines
+395
to
+397
| shell_audit = review_dir / "shell-audit" | ||
| pngs = len(list(shell_audit.glob("*.png"))) if shell_audit.is_dir() else 0 | ||
| check("P1-shell", True, pngs >= 8, f"shell-audit PNG count={pngs} (plan: 8)") |
There was a problem hiding this comment.
In a clean checkout that does not have the author’s ~/.reframe-review/shell-audit directory, this invariant always counts 0 PNGs and makes python docs/validation/tools/verify_ssot_claims.py exit 1 even though this commit tracks the shell-audit PNGs under docs/plans/v1.5/shell-audit. Since the verifier is now committed as the reproducible SSOT check, using a machine-local pre-land path makes it non-reproducible for CI/other developers; point this check at the tracked shell-audit directory or mark it non-gating like P1-corpus.
Useful? React with 👍 / 👎.
Comment on lines
+26
to
+30
| ROOT = Path(__file__).resolve().parent.parent | ||
| TARGET = sys.argv[1] if len(sys.argv) > 1 else "4.14.0.94" | ||
|
|
||
| # The attribute surface, derived from the source rather than hand-listed. | ||
| SRC = ROOT / "sidecar/media_studio" |
There was a problem hiding this comment.
Because this script now lives under docs/validation/tools, parent.parent resolves to docs/validation, so SRC becomes docs/validation/sidecar/media_studio and no sidecar files are scanned. In the dependency-bump probe scenario the script can still succeed after generic image ops while checking zero of the cv2.* attributes actually used by the sidecar, defeating the wheel-level guard described here; derive the real repo root before building SRC.
Useful? React with 👍 / 👎.
Comment on lines
+57
to
+58
| "C1a", | ||
| "C1b", |
There was a problem hiding this comment.
C1a is the actual speechbrain==1.1.0 pyproject pin, not a known-stale doc, but listing it in OPEN_ITEMS means check() treats any mismatch as non-failing. If that pin drifts or disappears, this verifier exits 0 and labels the regression as NOW-FIXED, so the committed SSOT guard cannot catch exactly the dependency-pin drift it claims to validate; keep only genuinely stale assertions in this set and make fixed invariants fail on mismatch.
Useful? React with 👍 / 👎.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The documentation reconciliation. Three things: the v1.5 authority corpus was untracked on one machine, the audit ledger did not keep its own promises, and ten documents asserted things the code had already changed.
Every factual claim behind this was checked mechanically before anything was edited —
docs/validation/tools/verify_ssot_claims.py, 40 claims. Three came back mismatched and all three were my verifier's bugs, not the plan's: a comment that wraps across lines (PINNED\n# to 1.0.3), and two use-vs-mention errors where a name appears only inside a comment explaining its own absence or inside a test regex alternation. Fixed, then 40/40.R1 — Authority lives in the repo, or it is not authority
14 documents (~440 KB) and 8 shell-audit screenshots lived only in
~/.reframe-review/. They were never reviewable, never diffable, and one disk failure from gone. They land verbatim so the corrections arrive as diffs. Two are archived rather than promoted, one because it is actively wrong in an instructive way: it asserts "there is NO 'YuNet' anywhere in the repo (zero matches)" — there are 369 across 44 files, and the detector it contradicts had landed ~24 h before that file's mtime. A confident "zero matches" from a stale working copy reads exactly like a finding..gitignore: one narrowing, and one deliberate refusal to narrow.claude/→.claude/*+ negations. A trailing-slash directory pattern makes every!inert, because git cannot re-include a file whose parent directory is excluded. Measured: a brand-new.claude/commands/x.mdresolved to.gitignore:63and was silently dropped, while six already-tracked files kept working. That split is invisible until someone adds a command and it never lands..audit/stays fully ignored — the migration plan wanted it narrowed, and I did that, then undid it. Its scripts did need to escape a directory declared disposable (git clean -xfdwas destroying the regeneration recipe for the tracked ledger, i.e. the exact property that made deleting the derived bytes safe). But un-ignoring them in place would have tracked each in two locations — the duplication this pass exists to remove. They moved todocs/validation/tools/instead.Verified both directions, 14 probes: every path that must become trackable does, all ~10 MB of derived bytes stays ignored, and no already-tracked file becomes ignored.
The ledger keeps its promises now
:27advertised| REFUTED | ... | 94 |and contained zero of them — so the one thing that row exists for, stopping a refuted finding from being re-raised, did not work. All 94 appended, count- and severity-checked against the advertised 4-critical / 90-high split.:57pointed at "the untracked full ledger" → nowv15-audit-ledger-unverified.md.gz, tracked, 6.66 MB → 2.26 MB, round-trip verified. Sole copy of 2348 findings, no longer on one machine.Ten contradictions, every one resolved in the code's favour
Detail is in the commit messages. The two that mattered:
SpeechBrain —
pyproject.tomlsaid "PINNED to 1.0.3 … 1.1.0 is DELIBERATELY AVOIDED" six lines above"speechbrain==1.1.0". The plan called this BLOCKING and expected a live install to settle it. It didn't need one:diarize_backend.pyprobes for 1.1.0'sFetchConfig, andpinned_fetch_kwargs's own docstring states that emitting the 1.0.x shape "would break the diarizer outright". The pin is right; the prose and three docs were stale. The stated avoidance mechanism is also unreachable — measured against both real wheels,integrations/k2_fsais absent in 1.0.3, present in 1.1.0, and only touched on explicit attribute access vialazy_export_all.Chatterbox torch — the
.txtsaid2.11.0+cu128, the code tuple2.10.0+cu128.manager.py:617compares those lists to decide "installed", so the divergence meant a correctly-built env re-downloaded ~7.5 GB every time and nothing failed loudly. The.txtnow follows the tuple, and a new test compares the two lists — two sibling tests asserted the literals and the order but nothing compared them, which is why it stayed invisible. Both-states verified: re-introducing the desync turns the new test red.I declined one plan item: it says
main.ts/security.tscite the dead CONTRACTS clause. They cite §0/§7 for "no auth, no network servers, no telemetry" — all still true.Plan corrections found while executing it
AUDIT-LEDGER-TIERED.mdwas called delete-safe with "zero unique content". Measured against RECOVERED — the file the claim is actually about; comparing it to the tracked ledger, as my first pass did, measures a pair nobody claimed anything about — it holds 8 blocks absent from both. Not delete-safe; stays untracked and undeleted.AUDIT-LEDGER-FINAL.mdis a duplicate as claimed (131/131 present; the one "differing" block is an artifact of the REFUTED append). Neither file is archived into the repo: with the unique content extracted, archiving 1.17 MB of verified duplicate would recreate the second source of truth.The verifier is now a regression guard, not a snapshot
Two defects in the tool this pass promoted. It hardcoded
ROOT = parent.parent, correct at its authoring location and wrong after the move — it reported 26 of 36 claims mismatched, blaming the repo for its own relocation. And half its claims asserted the broken state, so green would have meant "still broken" and fixing a doc would have turned it red. Claims are now split 29 invariants / 11 open, with open ones flaggedNOW-FIXED (retire it from OPEN_ITEMS)when they flip — which is how the 7 corrections in this PR became permanent guards.Also: three real e2e fixes verified in CI
Dispatch 30677298418 confirmed the earlier round worked — golden-journey passes on Windows,
e2e-sidecaris green (was 2 failed), and the VISUAL baseline REGEN step ran for the first time. It surfaced three newpreview.spec.tsfailures, identical on Windows and macOS: one unpinned-viewport geometry assertion, which then leakedadvancedOpen=trueinto the next two tests because the restore lived at the end of a body that aborted. Both fixed here; the viewport fix reuses the visual suite's own exported constants.Gate
pre-commitall Passed ·tsc0 ·tsc -p tsconfig.e2e.json0 ·basedpyright0 errors · vitest 100% (19733 stmts / 6583 branches / 1160 fns) · pytest 100%, 6108 passed ·charter_checkOK, 6 gates consistent ·verify_ssot_claims40/40.