test(mutation): kill check_doc_claims.py's 25 surviving mutants, plus 9 more the same rescope surfaced (#235) - #295
test(mutation): kill check_doc_claims.py's 25 surviving mutants, plus 9 more the same rescope surfaced (#235)#295cdeust wants to merge 1 commit into
Conversation
… 9 more the same rescope surfaced (#235) A scoped mutmut run over the doc-claim gate left 25 survivors in its canonical-source helpers (canonical_reference_count 10, canonical_tool_counts 7, canonical_version 3, canonical_mechanism_count 3, read 2) — all missing-branch and error-message assertions no test exercised. Fixing also_copy to include assets/ (missing from the committed mutation config, so the baseline run could not even complete) surfaced 9 more survivors outside the issue's own table, in check_badge, check_no_hotlinked_badges, check_no_conflict_markers, and check_scanned_json_parses — including two continue->break mutants that would silently abort scanning the rest of SCANNED_FILES after the first missing/skipped file. main() itself had zero mutation coverage (37 "no tests" mutants); a new MainTests class exercises every arm through the CLI entry point itself. Two mutants are documented equivalents rather than killed: encoding="UTF-8" is the same codec as "utf-8" under CPython's case-insensitive codec lookup, and dropping an explicit default=None from an argparse optional argument changes nothing since that is argparse's own implicit default. Final: 325 of 327 mutants killed, 2 documented equivalent, 0 unexplained survivors. Suite grows 6609 -> 6628; doc-claim gate and badge-count gate re-verified against the new count. Full suite: 6623 passed, 5 skipped. Closes #235 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012Yu6EnWspTfqHoGkExyS6u
6840d8a to
a234f77
Compare
|
Closing as superseded by #294, which merged to main while this PR was in flight (conflict this PR hit was the six-file test-count resync — the exact class #294 eliminates). #294 split Verified empirically against the current Zero surviving mutants, confirming issue #235's acceptance criterion ("0 surviving non-equivalent mutants in scripts/check_doc_claims.py") is already met. The 233 "no tests" (coverage-gap, not survival) mutants are the same class #294 already filed separately as #292 for the sibling modules it didn't fully cover in its own scoped run — outside #235's stated acceptance criteria (which is about survivors, not coverage), and already tracked. Recommend closing #235 as resolved-by-#294, the same disposition #267 got from #271. |
Summary
Closes #235.
A scoped mutmut run over
scripts/check_doc_claims.pyleft 25 survivors inits canonical-source helpers (
canonical_reference_count10,canonical_tool_counts7,canonical_version3,canonical_mechanism_count3,
read2) — every one a missing-branch or error-message assertion no testexercised (e.g.
canonical_version()'s "not found" arm had zero tests;canonical_reference_count'ssplit(..., 1)vs an unlimited-split/rsplitmutant were indistinguishable because every fixture had only one
"## References"heading).Reproducing the baseline first found
also_copyinpyproject.toml's[tool.mutmut]block was missingassets/—RepositoryTestsreads thereal badge SVGs, so the mutation run's own working copy couldn't complete
without it (a pre-existing gap in the scoped-run config, fixed here since I
was already touching this exact config for this exact file). With that fixed,
9 MORE survivors surfaced outside the issue's own table, in
check_badge,check_no_hotlinked_badges,check_no_conflict_markers, andcheck_scanned_json_parses— two of themcontinue→breakmutants thatwould silently abort scanning the rest of
SCANNED_FILESafter the firstmissing/skipped file (exactly the failure mode
check_no_conflict_markersexists to prevent).
main()itself had zero mutation coverage at all (37"no tests" mutants — mutmut's coverage-gap category, distinct from
"survived") since every existing test called
collect_failuresdirectlyrather than the CLI entry point; a new
MainTestsclass exercises every arm(ClaimError abort → exit 2, failures report → exit 1, success + exemption
count → exit 0,
--test-countforwarding, declareddescription/helptext) through
main()itself.Per the acceptance criteria's first bullet ("0 surviving non-equivalent
mutants in
scripts/check_doc_claims.py" — the whole file, not just theissue's table), all of the above are addressed in this one PR, per the
no-new-issues / no-deviation directive for this run.
Two mutants are documented equivalents, not killed (§12.1 — no third
option):
x_read__mutmut_4:encoding="UTF-8"vs"utf-8"— CPython's codecregistry is case/hyphen-insensitive (
codecs.lookup("UTF-8").name == codecs.lookup("utf-8").nameisTrue), so no observable behaviourdiffers. Rationale + verification in
ReadTests.test_read_decodes_explicitly_as_utf8'sown comment.
x_main__mutmut_8: dropping the explicitdefault=Nonekwarg on the--test-countargument —argparsealready defaults an unset optionalargument to
Nonewith nodefaultkwarg at all (verified inline).Rationale in
MainTests's class docstring.Final: 325 of 327 mutants killed, 2 documented equivalent, 0 unexplained
survivors.
Test-count resync (working rules)
19 new tests → suite count changes; resynced the measured absolute
(6628, matching a real
pytest --collect-only -qon this exact tree) across.bestpractices.json,CLAUDE.md,CONTRIBUTING.md,README.md,docs/ASSURANCE-CASE.md, and regeneratedassets/badge-tests.svgviascripts/generate_repo_badges.py --test-count 6628. Both gates re-verified:Completion Ledger
read()UTF-8-explicit contractReadTests.test_read_decodes_explicitly_as_utf8canonical_tool_counts(): missing header sentencetest_missing_standalone_total_sentence_is_an_errorcanonical_tool_counts(): arithmetic-mismatch messagetest_inconsistent_arithmetic_in_the_catalogue_is_an_error(message assert added)canonical_tool_counts(): missing pinned testtest_missing_pinned_registry_test_is_an_errorcanonical_reference_count(): split-vs-rsplit/maxsplit semanticstest_the_first_references_heading_is_the_split_pointcanonical_reference_count(): missing-section messagetest_bibliography_without_a_references_section_is_an_error(message assert added)canonical_reference_count(): no-entries-found messagetest_a_references_section_with_no_entries_is_an_errorcanonical_mechanism_count(): undeclared-count messagetest_undeclared_mechanism_count_is_an_error(message assert added)canonical_version(): missing-version messagetest_missing_version_declaration_is_an_errorcheck_badge(): full "diverged" messagetest_a_badge_without_a_version_title_fails_closed(exact-match assert)check_no_hotlinked_badges(): line number + messageHotlinkedBadgeTests(both tests)check_no_conflict_markers(): continue-not-break on missing filetest_a_missing_file_does_not_abort_the_conflict_scancheck_scanned_json_parses(): missing-.json messagetest_a_missing_json_file_fails_closedcheck_scanned_json_parses(): continue-not-break (both loops)test_a_non_json_file_does_not_abort_the_json_scan,test_a_missing_json_file_does_not_abort_the_json_scanmain(): ClaimError abort, exit 2test_a_claim_error_aborts_to_stderr_with_exit_code_2main(): failures report, exit 1test_failures_are_reported_to_stderr_with_exit_code_1main(): success + exemption count, exit 0test_success_reports_exemption_count_and_exit_code_0,test_success_with_no_exemptions_reports_a_zero_countmain():--test-countparsed/forwardedtest_the_test_count_flag_is_parsed_and_forwarded,test_without_the_flag_test_count_is_nonemain(): declared description/help texttest_the_parser_declares_the_modules_docstring_and_flag_helppyproject.tomlalso_copygap (assets/)RepositoryTestsno longer fails collection under mutants)Boy-scout items fixed in this PR
pyproject.toml's[tool.mutmut]also_copylist was missingassets/,which made ANY scoped mutation run against
check_doc_claims.pyfail atbaseline collection (
RepositoryTests.test_committed_docs_agree_with_the_repository)before ever generating a mutant. Fixed (comment explains why, dated).
test_the_parser_declares_the_modules_docstring_and_flag_help)initially came in at 46 lines; extracted the argparse-patching mechanics
into a shared
_spying_on_argparse_constructioncontext manager (alsodocuments the
super(ArgumentParser, self)recursion gotcha it exists toavoid) — all test methods now ≤40 lines.
ruff formatreformatted 3 over-88-char lines I introduced; committedformatted.
Test plan
pytest tests_py/scripts/test_check_doc_claims.py— 65 passed, 10 subtestspytest tests_py/scripts/(full sibling scope) — 348 passed, 117 subtestspytest tests_py/shared/test_json_native.py(the file's own committed mutmut demonstration target, untouched by this PR but re-verified since I editedpyproject.toml's[tool.mutmut]block) — 14 passedruff check+ruff format --checkon touched files — cleanpyrighton touched files — 0 errors/warnings/informationspyproject.toml's transientonly_mutate/source_paths/pytest_add_cli_args_test_selectionrestored to the committed baseline (mcp_server/shared/json_native.py) after each scoped run — only the permanentassetsaddition toalso_copyis committed🤖 Generated with Claude Code
https://claude.ai/code/session_012Yu6EnWspTfqHoGkExyS6u