Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ adheres to [Semantic Versioning](https://semver.org/).
- **`.gitattributes` marks `fuzz/corpus/**` as binary**: end-of-line normalisation would have rewritten the CRLF corpus seed to LF on checkout, silently deleting the case that seed exists to cover.
- **`.bestpractices.json` was committed carrying four unresolved merge-conflict blocks**, which left it invalid JSON — and the file is transcribed into the OpenSSF Best Practices questionnaire, so an unparseable copy is a broken consumer rather than a stale number. It passed the doc-claim gate, CodeQL and 18 green checks, because `.bestpractices.json` is one of that gate's own `SCANNED_FILES` and every check it runs is a claim regex: a regex matches the first side of a conflict and never looks at the file's structure. Both sides of all four blocks were byte-identical, so the repair is lossless (verified by comparing the sides, not by choosing one). The gate now also runs `check_no_conflict_markers` and `check_scanned_json_parses`, both derived from `SCANNED_FILES` so a newly scanned file is enrolled with no further edit, and both failing closed on a file they cannot read. Only the **labelled** markers are matched (`<<<<<<< HEAD`, `>>>>>>> origin/main`) — a bare `=======` is a legal setext H1 underline in Markdown, and most scanned files are Markdown, so matching it would fail honest documents; a test pins that.
- **Any two PRs that added tests conflicted on six files, by construction — eliminated at the root** (#293). The collected test count was hand-carried as an exact figure in `.bestpractices.json`, `CLAUDE.md`, `CONTRIBUTING.md` (×2), `README.md` (×2) and `docs/ASSURANCE-CASE.md`, each checked for EQUALITY against whichever branch's own live `pytest --collect-only` count ran in CI. That count is a property of the post-merge tree, not of any one branch: two branches that each add tests compute two different, both-true numbers and must each edit the same six lines to match, so the second to merge silently overwrites the first's correct figure with its own now-stale one — measured on this repo as two red `main` runs (PR #280 synced to its own total, #278 added more tests against a stale base) and a PR rebased three times solely to resolve the resulting conflicts. `assets/badge-tests.svg` is now the ONLY artifact stating an absolute count; the five others point at it instead of restating the figure. The badge's own check moves from an exact match to a monotone **floor** (`doc_claim_structural.check_badge_floor`, `generate_repo_badges.stale_tests_badge`): a committed count that lags the live one is stale-but-true and passes, so a PR that only adds tests never touches it, and only an actual OVER-claim — a hand-typed number, or tests removed below what was claimed — fails. A standing regression guard (`test_no_prose_file_states_the_suite_size_any_more`) asserts no scanned file, including `.bestpractices.json`, states this claim in prose again. `check_doc_claims.py` (420 lines) and `generate_repo_badges.py` (305 lines) were both over the repo's 300-line file cap before this change needed to touch them further; split into `doc_claim_sources.py`/`doc_claim_scan.py`/`doc_claim_structural.py`/`repo_badge_catalog.py` (Extract Module) with zero behavior change, verified by an unchanged existing test suite before the floor logic was added.
- **`doc_claim_scan.py`, `doc_claim_structural.py`'s remaining functions and `repo_badge_catalog.py` — every mutant reported "no tests"** (#292), the last siblings in the `badge_render`/`check_badge_floor`/`doc_claim_sources` defect family (#262/#280/#293, and #235's own instance of it): `check_doc_claims.py` bare-imports the first two (`import doc_claim_scan`, `import doc_claim_structural`) and `generate_repo_badges.py` bare-imports the third, and a function's `__module__` is fixed at definition time to whatever name it was imported under (`mutmut/mutation/trampoline.py`, `module != decorated_func.__module__`) — never mutmut's dotted, path-derived `"scripts.<name>"`, so its trampoline never activated and every mutant in these three files showed "no tests" despite being exercised by real passing tests through the bare-imported path. Fixed the same way as the existing `check_badge_floor`/`doc_claim_sources` precedent: each sibling is loaded a second time via `importlib.util.spec_from_file_location("scripts.<name>", ...)`, and new direct-test classes (`DocClaimScanDirectTests`, `StructuralDirectTests` in `test_check_doc_claims.py`; `RepoBadgeCatalogDirectTests` in `test_generate_repo_badges.py`) call through those dotted references so mutmut's trampoline attributes the mutant to a real test. Verified with a real tally, not the absence of an error: a scoped `mutmut` run (`scripts/mutation_check.sh`) against `doc_claim_structural.py`, `doc_claim_scan.py`, `doc_claim_sources.py` and `repo_badge_catalog.py` plus both test files reports **301 mutants, 301 killed, 0 "no tests", 0 survived** — every assertion tightened to exact `assertEqual` (not `assertIn`) on the full message/dict, plus targeted `continue`-vs-`break` tests for multi-file scans for the loop-shaped survivors exact-match assertions alone don't reach. Boy-scout: `check_scanned_json_parses`'s `FileNotFoundError` branch (a missing scanned `.json` file) had no test at all, direct or indirect, before this change. Test-only; no production code changed.

### Added
- **The README's repo-derived badges are now self-hosted SVGs, gated in CI** — `assets/badge-{license,python,tests,references,version}.svg`, `scripts/generate_repo_badges.py`, `scripts/badge_render.py`. Five hotlinked `img.shields.io` images became committed files, for the reason the MCP Toplist badge already was one: a remote badge URL is a third-party request fired on every README view, and it lets its host restate our claim with no commit in this repository. **Only repo-derived figures were converted**, and the line is deliberate — every one of these (licence, Python floor, collected test count, bibliography size, package version) is determined BY THIS REPOSITORY, so a committed copy can always be made true again from the working tree with no network access. That is why they are kept honest by a **blocking `--check` gate on every push and PR** rather than by a cron: drift is caught where it is introduced. **Two badges were deliberately NOT converted and must not be:** the CI status badge reports the LIVE result of the last run on main, so a committed copy would assert "passing" while main was broken — a static build-status badge is a false claim by construction, not merely a stale one — and it is GitHub-hosted, so it is not a third-party beacon in the first place; the OpenSSF Best Practices badge reflects an external body's live assessment that can be downgraded without any commit here, and `.bestpractices.json` separately justifies displaying THEIR badge image. The MCP Toplist badge sits between the two and stays committed because it carries an explicit "as of <month>" stamp, which keeps a stale copy a true statement about a point in time. **The conversion had to rewire the gate it would otherwise have silenced:** `check_doc_claims.py` enforced the version and test-count claims by regex over the shields.io URLs (`badge/version-(\d+\.\d+\.\d+)`, `badge/tests-(\d+)_passing`), so self-hosting alone would have left both patterns matching nothing while the gate still reported success. They now read the figure out of the committed SVG's own `<title>` and **fail closed** on a missing file or an unmatched title — the predecessor's `if badge and ...` passed silently the moment its subject disappeared. A new check also fails any reintroduced `img.shields.io` hotlink in the README, so reverting this is loud rather than quiet. Found by the new tests before shipping: `--` is illegal inside an XML comment, and the first cut described its own gate as `--check` and its source as `pytest --collect-only` in the provenance comment, leaving **all five badges unparseable**; the renderer now parses every badge it produces and refuses to return one that is not well-formed, so no future wording can reintroduce the class. The shared renderer was extracted from the MCP Toplist refresher first, as a separate behavior-preserving commit whose proof is that `assets/badge-mcp-toplist.svg` is byte-identical and its 51 tests pass unchanged. Suite grows 6348 → 6373.
Expand Down
Loading