Skip to content

mutmut: sibling-imported scripts/ modules show every mutant as 'no tests' (bare-import __module__ mismatch) #292

Description

@cdeust

What

scripts/doc_claim_scan.py, scripts/doc_claim_sources.py, and scripts/repo_badge_catalog.py (and, before this was fixed for it, scripts/doc_claim_structural.py) are extracted sibling modules that scripts/check_doc_claims.py/scripts/generate_repo_badges.py import bare (import doc_claim_scan, etc. — the same sys.path-based pattern this repo already uses for badge_render/check_doc_claims). When mutation-tested, every mutant in these files reports no tests, even though their functions are exercised by passing tests via the bare-imported path.

Root cause (confirmed by reading mutmut/mutation/trampoline.py)

wrap_in_trampoline's dispatcher compares module != decorated_func.__module__, where module comes from the MUTANT_UNDER_TEST env var mutmut sets using the dotted, path-derived name (e.g. "scripts.doc_claim_structural"). A function's __module__ is set by Python at definition time to whatever name the module was imported under. Bare-imported (import doc_claim_structural), every function in that file carries __module__ == "doc_claim_structural" — never equal to the dotted "scripts.doc_claim_structural" mutmut expects — so the trampoline always falls through to the original implementation and stats/mutant activation never attribute to any test. This is the exact defect class tests_py/scripts/test_generate_repo_badges.py already documents and works around for badge_render.py (see its "Loaded under its own dotted name" comment) — it just hadn't been hit yet for the newer doc_claim_*/repo_badge_catalog split (issue #287).

Fix, already applied for one file as a demonstration

tests_py/scripts/test_check_doc_claims.py now also loads doc_claim_structural.py a second time via importlib.util.spec_from_file_location("scripts.doc_claim_structural", ...), and CheckBadgeFloorDirectTests calls check_badge_floor through THAT reference directly. Re-running the scoped mutation check confirmed: doc_claim_structural.py's "not covered" mutants dropped from 12 to 0, and this surfaced two genuine additional survivors (a message-text assertion that only checked a prefix), now fixed.

Remaining scope

Apply the same dotted-load + direct-test pattern to:

  • scripts/doc_claim_structural.py's remaining functions (check_badge, check_no_hotlinked_badges, check_no_conflict_markers, check_scanned_json_parses) — currently covered only indirectly via gate.collect_failures's bare-imported copy (real behavioural coverage, zero mutation coverage).
  • scripts/doc_claim_scan.py (scannable_lines, exemption_registry, scan_claims, check_counts) and scripts/doc_claim_sources.py (canonical_tool_counts, canonical_reference_count, canonical_mechanism_count, canonical_version) — not included in the mutation-testing scope this issue's originating PR ran (only check_doc_claims.py, doc_claim_structural.py, generate_repo_badges.py were scoped), so their "no tests" status wasn't directly observed this pass, but the same bare-import mechanism applies and should be assumed affected until verified.
  • scripts/repo_badge_catalog.py (fixed_badge_specs, tests_badge_spec) — same mechanism; not yet scoped into any mutation run.

Repro

scripts/mutation_check.sh's rewiring, applied to only_mutate = ["scripts/doc_claim_structural.py"],
pytest_add_cli_args_test_selection = ["tests_py/scripts/test_check_doc_claims.py"],
then: mutmut run; mutmut results  # every x_check_badge*/x_check_no_*/x_check_scanned_json_parses* mutant: "no tests"

Filed per coding-standards.md §15.1 (scoped to its own fix rather than folded into the originating PR, which only needed check_badge_floor's own attribution corrected) — root-caused during mutation-testing scripts/check_doc_claims.py's issue #287 floor-check logic.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions