fix(ast-parser): degrade gracefully on tree-sitter grammar DownloadError

[cdeust]

Summary

main is red: CI run 30592244731 (Test (Python 3.10), 2026-07-31) failed with an
uncaught tree_sitter_language_pack.DownloadError — tests_py/benchmarks/test_codebase_alteration.py
(5 tests) and tests_py/core/test_ast_extractors.py::test_decorated_function.

tree-sitter-language-pack is a declared dependency, but it resolves each grammar's
shared library lazily over the network at get_parser() call time, not at install
time (measured: an already-cached language returns in <50ms with no network attempt;
an uncached one takes a real round trip against
https://github.com/xberg-io/tree-sitter-language-pack/releases). ast_parser.py
already treated a missing pack (ImportError) as a handled degraded mode (its own
comments say so), but had no handling for a pack that imports fine and then fails to
fetch a grammar — an offline install, air-gapped environment, proxy, or upstream
outage reached get_parser(language).parse(content) uncaught, so codebase_analyze
raised a third-party exception type instead of degrading. Same defect class this repo
has already been bitten by twice (FlashRank silent-absence; MCP stdio response-loss,
#284): a degraded path existed and did not cover the failure that actually happens.

Root cause (Move 4)

• Symptom: codebase_analyze crashed with an uncaught DownloadError instead of
  degrading, whenever a grammar's shared library couldn't be fetched over the network.
• Architectural cause: _get_extractor_and_tree treated only "pack not installed"
  as a handled degraded-mode boundary; it never accounted for the pack's own lazy
  network-fetch-at-call-time design, so a network failure inside
  get_parser().parse() propagated a third-party exception straight through
  parse_file_ast to the caller.
• Fix + correctness argument: catch DownloadError specifically around the exact
  call site, returning the same None sentinel parse_file_ast already branches on
  for the regex fallback — the postcondition ("never propagates a third-party
  exception; always returns a valid degraded result") holds by construction, verified
  by 3 regression tests that fail pre-fix / pass post-fix (including the log signal
  itself), a scoped mutation run (0 surviving non-equivalent mutants on the changed
  function), and a full 7017-test suite green with no regressions.

Reproduction (before the fix)

monkeypatch tree_sitter_language_pack.get_parser to raise DownloadError
>>> parse_file_ast("demo.py", b"def f():\n    pass\n")
tree_sitter_language_pack.DownloadError: Download error: Failed to fetch manifest
from https://github.com/xberg-io/tree-sitter-language-pack/releases

Uncaught, propagates straight out of _get_extractor_and_tree / parse_file_ast.

Changes

1. mcp_server/core/ast_parser.py — _get_extractor_and_tree catches
   DownloadError (not the pack's broader Error base — only DownloadError is
   evidenced) around get_parser(language).parse(content), returns None (same
   degraded-mode contract as the ImportError branch), and logs one actionable
   logger.warning naming the language and reason on every occurrence (not
   suppressed after the first — this runs once per file, not once per process like a
   singleton model load, so suppressing repeats would hide a mid-run outage).
2. tests_py/core/test_ast_parser_language_contract.py — 3 new regression tests,
   all verified to fail on the pre-fix code:
   • test_download_error_falls_back_instead_of_raising — the degraded-mode return.
   • test_download_error_logs_language_and_reason — asserts the exact log
     message (both static halves + the dynamic language/reason substitutions), not
     merely a substring — this is what a scoped mutation run demanded (see below).
   • test_download_error_degrades_through_parse_file_ast — the public entry point
     codebase_analyze actually calls, end-to-end.
3. .github/workflows/ci.yml — test and test-sqlite jobs (the two that
   install the codebase extra) gain a Resolve tree-sitter cache directory +
   Cache tree-sitter grammars + Prefetch tree-sitter grammars step trio, same
   cache-then-retry-with-backoff shape as the existing HF-embedding/FlashRank steps,
   for the same reason: fetch every language in AST_SUPPORTED (read from the
   module, not hand-copied, so it cannot drift) once, with retries, before pytest
   starts — the suite's own pre-existing direct get_parser(...) calls in
   test_ast_extractors.py/test_ast_parser_language_contract.py then never touch
   the network mid-run. Chose CI-side provisioning over touching those call sites
   directly: the root cause was mid-test-run network non-determinism, not the test
   assertions themselves, and this mirrors the exact established convention already
   proven in this file for the identical class of problem.
4. pyproject.toml [tool.mutmut] also_copy — adds "scripts". Discovered
   while mutation-testing this change: tests_py/conftest.py (the root conftest
   every test in the suite collects through) does
   from scripts.check_venv_lock_parity import postgresql_extra_drift at module
   level, and without this entry any scoped mutation run — not just this one —
   fails collection with ModuleNotFoundError: No module named 'scripts' before
   scoring a single mutant. Small, additive, unblocks the tooling repo-wide.
5. CHANGELOG.md — entry under [Unreleased] / Fixed.

No other unguarded get_parser/get_language call site exists in mcp_server/
(swept repo-wide with grep -rn "get_parser(\|get_language("); ast_parser.py line 99
was the only production call).

Completion Ledger

Path / item	Evidence
DownloadError during get_parser().parse() → degraded None	test_download_error_falls_back_instead_of_raising; fails pre-fix (reproduced)
Degraded-mode signal emission (language + reason, exact wording)	test_download_error_logs_language_and_reason — exact-string match, kills the 3 mutants a substring-only assertion left surviving
Public entry point (parse_file_ast/codebase_analyze) never propagates the exception	test_download_error_degrades_through_parse_file_ast; fails pre-fix
ImportError (missing pack) degraded mode — pre-existing, unchanged	test_missing_pack_falls_back_instead_of_raising, test_is_available_is_false_when_the_pack_is_absent — still 16/16 green in the file
Happy path (parser obtained, tree parsed) — unchanged	test_supported_language_with_an_extractor_parses, full test_ast_parser*/test_ast_extractors.py suite (64 tests) green
CI network non-determinism (mid-suite get_parser calls)	test/test-sqlite jobs' new prefetch trio; local differential measurement: cached language <50ms/no network, uncached ~150ms/real fetch (elixir, outside AST_SUPPORTED, confirmed absent from downloaded_languages() beforehand)
Mutation strength on the changed function	Scoped scripts/mutation_check.sh run: 157 mutants generated for ast_parser.py against the file's full test selection; 0 surviving mutants in _get_extractor_and_tree (my changed function) after the message-assertion strengthening; 22 pre-existing survivors remain in _node_text/_extract_module_doc — untouched by this diff, already documented out-of-blast-radius by issue #269/PR #286 for this exact file
Full suite, no regressions	pytest -q (offline env vars matching CI): 7012 passed, 5 skipped, 121 subtests passed, twice (before and after the final test-assertion strengthening)
Lint / format	ruff check . / ruff format --check . — clean, repo-wide
Type check	pyright mcp_server/core/ast_parser.py — 0 errors (sanity-checked against a deliberately-wrong type to confirm it's really resolving tree_sitter_language_pack, not silently no-op)
Doc-claims / badges	check_doc_claims.py --test-count 7017 / generate_repo_badges.py --check --test-count 7017 — both clean (the tests badge is a floor, not exact-match, per issue #293 — no regeneration needed)
CI workflow syntax	actionlint .github/workflows/ci.yml (shellcheck-integrated) — 0 findings
No conflict markers / .bestpractices.json parses	checked directly — clean

Stakes classification (objective)

High — ast_parser.py is imported by 7 other modules (>5 threshold) and has 2
distinct git log --format='%an' authors in the last 90 days; both criteria
independently place it High per the standard's objective test. Full discipline
applied: reproduction before fix, root-cause fix (not a band-aid), regression tests
including signal-emission assertions, scoped mutation testing on the changed
function, full-suite verification, and CI-hermeticity fix rather than a test skip.

Working notes

Detached worktree at Cortex-fix-grammar-fetch-degraded, branch
fix-grammar-fetch-degraded, based on origin/main @ 469e70c2 (unchanged
throughout). Left open per instructions — not merging; the coordinator merges and
verifies main post-merge. Not touching lint-finish-239 (separate in-flight branch
noted as off-limits).

🤖 Generated with Claude Code