From 1afae0acd89fdb7c02ff1622f64aeb462b386c53 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 5 Aug 2026 11:17:53 +0000 Subject: [PATCH] fix(health): give the conductor a STALE branch (#198) The health conductor had no STALE branch, so a stale-only readiness verdict rendered as UNKNOWN. The card printed "adopted verdict = stale" with a score, then the recommendation printed "UNKNOWN - could not obtain a verdict from the vitals faculty", the counts read 0/0/0, and the script exited 4 - the same code as "Heart unreachable". PyAutoHeart makes STALE a first-class freshness tier and AUTONOMY.md leg 4 treats it as PASSING the dev-ship gate, so collapsing it onto unknown is not a cosmetic slip: a machine caller cannot tell an evidence gap from a dead sensor. Three sites, all in agents/conductors/health/health.sh: * The triage item classifier walked only (red, yellow), so stale reasons never became items. It now walks stale_reasons too, and - mirroring the existing "severity wins over the keyword class" rule for red - forces stale reasons to a new `evidence-gap` kind. They are action items (re-run the named check), so they must not land in the bucket the renderer labels "accept, not action items". Each carries a real refresh entry point: `pyauto-brain release validate` for the hard readiness gate, `pyauto-heart verify_install`, or `pyauto-heart tick` for anything the <30s tick measures. A capability with no known entry point gets no command and is named in prose - never invented, the same discipline the file already applies to `pyauto-heart fix` topics. * The recommendation chain gained a STALE branch ahead of the baseline-gap one. It names the top evidence gap, prefers the release-validation gap when there is one (that is the leg that makes GREEN reachable), and states the tier's semantics: the remedy is a fresh run, never a code fix; a release still requires GREEN even though the dev-ship gate treats STALE as passing. * `_exit_code_for` maps stale to 6, not the free slot 1. 1 is the shell's own generic failure, and STALE is a passing tier for the ship gate, so a crash must never be readable as a pass - the same fail-safe reasoning the header already applies to usage errors (5, "so misuse is never read as a real YELLOW"). Existing codes are untouched: 0 green / 2 yellow / 3 red / 4 unknown / 5 usage. The header's exit-code table and the conductor's AGENTS.md (its second copy of that table, plus the triage taxonomy and the recommended-action table) are updated to match, with the rationale for 6 travelling alongside them. tests/test_health_conductor.py is new: 18 hermetic tests that stub pyauto-heart on PATH and drive the conductor end-to-end through the vitals faculty. They pin the three fixed sites, that every verdict maps to a distinct exit code, that the documented tables do not drift from _exit_code_for, that refresh commands are never invented, and - as regression guards - that red still dominates a stale reason and that a Heart with no stale_reasons key behaves exactly as before. Full suite: 222 passed. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01FbLrcRCmUvhRQdtZbDWMDD --- agents/conductors/health/AGENTS.md | 27 ++- agents/conductors/health/health.sh | 107 ++++++++++- tests/test_health_conductor.py | 278 +++++++++++++++++++++++++++++ 3 files changed, 399 insertions(+), 13 deletions(-) create mode 100644 tests/test_health_conductor.py diff --git a/agents/conductors/health/AGENTS.md b/agents/conductors/health/AGENTS.md index 738ef96..02819c0 100644 --- a/agents/conductors/health/AGENTS.md +++ b/agents/conductors/health/AGENTS.md @@ -40,7 +40,7 @@ mediation is the Brain reasoning layer on top of it. Each stage: ### Triage taxonomy -`health.sh` classifies every readiness reason into one of three kinds and ranks +`health.sh` classifies every readiness reason into one of four kinds and ranks them (most-blocking first). This is the split the loop reasons over: - **Real problems** — genuine health signals to act on now; they block green. @@ -51,10 +51,23 @@ them (most-blocking first). This is the split the loop reasons over: behind, and version skew have **no** `fix` shortcut — those are human / Release-Agent work (e.g. landing a dev branch back to main), not a health-conductor dispatch. +- **Evidence gaps** — every reason Heart files under its STALE freshness tier + (`stale_reasons`): nothing is known-bad, but some evidence is missing or + expired. Severity decides this, not the keyword class — the remedy is a fresh + **run** of the named check, never a code fix, which is exactly what separates + a stale reason from a yellow one. They *are* action items (unlike the accepted + baseline gaps below) and the triage cites the real refresh entry point: + `pyauto-brain release validate` (`validate` — the hard readiness gate, so it is + recommended ahead of the others), `pyauto-heart verify_install` + (`verify_install`), `pyauto-heart tick` (anything the <30s tick measures). + A capability with no known refresh entry point gets no command — named in + prose instead, never invented. - **Expected first-run gaps** — standing YELLOW unknowns you *accept*, not action items: "no test-run report" (`test_run`), "install verification not run" (`verify_install`), "no release validation for current source" (`validate`). - These are exactly what the `release validate` leg closes. + These are exactly what the `release validate` leg closes. (Heart files the same + three reasons under `stale_reasons` once it has the freshness tier — then they + arrive as **evidence gaps** above, and this bucket stays empty.) - **Advisory** — monitoring only; does not gate readiness (`worktree_drift`, `open_prs`, `url_check`). @@ -72,6 +85,7 @@ Deterministically, given the adopted verdict and triage: | **GREEN** | none — a release conductor may proceed | | **RED** (real blockers) | resolve the top blocker; cite `pyauto-heart fix …` where one exists, else flag it as human/Release-Agent work. **Do not** dispatch a release while RED — the release preflight (Stage 0/1) aborts on it anyway | | **YELLOW**, a real warning present | clear the warning (cite its `fix`), then re-assess | +| **STALE** (the freshness tier) | refresh the top evidence gap — `pyauto-brain release validate` when the release-validation report is one of them (the hard gate), else the capability's own refresh entry point. Nothing is known-bad, so this is never a code fix. A release still requires GREEN; the dev-ship gate (`AUTONOMY.md` leg 4) treats STALE as passing | | **YELLOW**, only baseline gaps | `pyauto-brain release validate` — the leg that flips "no release validation" and makes GREEN reachable | | **UNKNOWN** | `pyauto-brain vitals` to refresh, then re-run | @@ -104,8 +118,13 @@ For just the raw read (no loop), consult the faculty directly: `bin/pyauto-brain vitals`. Exit codes mirror the adopted verdict so a caller (and the loop) can branch: -`0` green · `2` yellow · `3` red · `4` unknown. A CLI usage error (unknown -subcommand) exits `5`, kept distinct so misuse is never read as a real YELLOW. +`0` green · `2` yellow · `3` red · `4` unknown · `6` stale. A CLI usage error +(unknown subcommand) exits `5`, kept distinct so misuse is never read as a real +YELLOW. STALE takes `6` rather than the free slot `1` for the same fail-safe +reason: `1` is the shell's own generic failure, and STALE is the tier +`AUTONOMY.md` leg 4 treats as **passing** the dev-ship gate — so a crash must +never be readable as a passing verdict. That separation is the whole point: a +machine caller can tell "Heart says STALE" from "Heart unreachable" (`4`). ## Prerequisites / caveats diff --git a/agents/conductors/health/health.sh b/agents/conductors/health/health.sh index a238e09..b7e65fd 100755 --- a/agents/conductors/health/health.sh +++ b/agents/conductors/health/health.sh @@ -49,9 +49,16 @@ # health.sh -h|--help # this header # # Exit codes mirror the ADOPTED verdict so a caller (and the loop) can branch: -# 0 green · 2 yellow · 3 red · 4 unknown. A CLI usage error (unknown +# 0 green · 2 yellow · 3 red · 4 unknown · 6 stale. A CLI usage error (unknown # subcommand) exits 5 — kept distinct from the verdict codes so misuse is never # read as a real YELLOW. +# +# STALE takes 6 rather than the free slot 1 for the same fail-safe reason: the +# shell's own generic failure is 1 (a missing `_common.sh`, a failed `readlink`), +# and STALE is the tier AUTONOMY.md leg 4 treats as PASSING the dev-ship gate — +# so a crash must never be readable as a passing verdict. 6 sits outside the 0-5 +# block and is never produced by bash or python here, which is what lets a +# machine caller tell "Heart says STALE" from "Heart unreachable" (4). set -uo pipefail @@ -123,6 +130,10 @@ score = r.get("score") ts = r.get("ts") red = list(r.get("red_reasons") or []) yellow = list(r.get("yellow_reasons") or []) +# The Heart freshness tier (PyAutoHeart/heart/readiness.py): nothing known-bad, +# some evidence missing or expired. A Heart that predates the tier emits no +# stale_reasons key at all, so this degrades to [] and behaves as before. +stale = list(r.get("stale_reasons") or []) # Optional: the set of capability ids Heart advertises, so triage stays honest # about the surface it maps to (reason over categories present in the manifest, @@ -144,6 +155,8 @@ if mpath and os.path.isfile(mpath): # Reason -> (capability, kind) by CATEGORY of signal. kind is one of: # real-problem a genuine health signal to act on now (gates green) +# evidence-gap a STALE-tier reason: the evidence is missing or expired and the +# remedy is RE-RUNNING the named check, never fixing code # baseline-gap an expected first-run unknown you accept (standing YELLOW) # advisory monitoring only; does not gate readiness # fix_topic is the `pyauto-heart fix ` class IFF Heart offers one for that @@ -166,9 +179,30 @@ RULES = [ (re.compile(r"\burl\b", re.I), "url_check", "advisory", None), ] -# Rank buckets: lower sorts first (most blocking first). -KIND_RANK = {"real-problem": 0, "advisory": 2, "baseline-gap": 3} -SEV_RANK = {"red": 0, "yellow": 1} +# Rank buckets: lower sorts first (most blocking first). An evidence-gap ranks +# below a real problem but above the accepted baseline gaps: it is actionable +# (re-run the check) and it is what stands between the organism and GREEN. +KIND_RANK = {"real-problem": 0, "evidence-gap": 1, "advisory": 2, "baseline-gap": 3} +SEV_RANK = {"red": 0, "yellow": 1, "stale": 2} + +# capability -> the REAL entry point that refreshes that evidence. Only genuine +# commands appear here (the same discipline as the fix topics above: never invent +# one). A capability absent from this map yields command None, and the +# recommendation names the check to re-run in prose instead. +REFRESH_CMD = { + # the release-validation report is a hard readiness gate; refreshing it is a + # DELEGATED leg (the release conductor owns the MCP GitHub boundary). + "validate": "pyauto-brain release validate", + "verify_install": "pyauto-heart verify_install", + # everything the <30s tick measures is refreshed by re-ticking Heart. + "repo_state": "pyauto-heart tick", + "version_skew": "pyauto-heart tick", + "worktree_drift": "pyauto-heart tick", + "script_timing": "pyauto-heart tick", + "ci_status": "pyauto-heart tick", + "open_prs": "pyauto-heart tick", + "unknown": "pyauto-heart tick", +} def repo_of(reason): # "PyAutoConf: on branch ..." -> "PyAutoConf"; else None. @@ -184,7 +218,7 @@ def classify(reason, severity): return "unknown", ("real-problem" if severity == "red" else "baseline-gap"), None items = [] -for severity, reasons in (("red", red), ("yellow", yellow)): +for severity, reasons in (("red", red), ("yellow", yellow), ("stale", stale)): for reason in reasons: cap, kind, fix_topic = classify(reason, severity) # Severity wins over the keyword class: anything Heart put in red_reasons @@ -194,6 +228,12 @@ for severity, reasons in (("red", red), ("yellow", yellow)): # and a real problem, even though both map to the `validate` capability.) if severity == "red" and kind != "real-problem": kind = "real-problem" + # The same rule on the freshness axis: anything Heart put in + # stale_reasons is an evidence gap by definition, whatever the keyword + # class says. Its remedy is re-running the check, so it is an action item + # -- NOT one of the standing baseline gaps you accept. + elif severity == "stale": + kind = "evidence-gap" repo = repo_of(reason) fix_cmd = None if fix_topic: @@ -210,8 +250,13 @@ for severity, reasons in (("red", red), ("yellow", yellow)): "capability": cap, "capability_known": (cap in known_caps) if known_caps else None, "kind": kind, - "blocks_green": severity == "red" or kind == "real-problem", + # An evidence-gap blocks green too: a release requires GREEN, and + # re-running the named check is precisely the path back to it. + "blocks_green": severity == "red" or kind in ("real-problem", "evidence-gap"), "fix": fix_cmd, + # Only an evidence gap carries a refresh entry point (None when the + # capability has no known one -- never invented). + "refresh": REFRESH_CMD.get(cap) if kind == "evidence-gap" else None, }) items.sort(key=lambda it: (KIND_RANK.get(it["kind"], 4), @@ -220,6 +265,7 @@ items.sort(key=lambda it: (KIND_RANK.get(it["kind"], 4), blockers = [it for it in items if it["severity"] == "red"] warn_real = [it for it in items if it["severity"] == "yellow" and it["kind"] == "real-problem"] +evidence = [it for it in items if it["kind"] == "evidence-gap"] gaps = [it for it in items if it["kind"] == "baseline-gap"] advisory = [it for it in items if it["kind"] == "advisory"] @@ -264,6 +310,40 @@ elif warn_real: "headline": "YELLOW — a genuine warning to act on: " + reason, "detail": "Maps to the " + cap + " capability. Clear it, then re-assess.", "checkpoint": CHECKPOINT} +elif verdict == "stale" or evidence: + # The freshness tier. Nothing is known-bad; some evidence is missing or + # expired, and the remedy is to RE-RUN the named check, never to fix code. + # Prefer the release-validation gap when there is one: it is the hard + # readiness gate, so refreshing it is the leg that makes GREEN reachable. + t = top([it for it in evidence if it["capability"] == "validate"]) or top(evidence) + if t is None: + rec = {"action": "refresh-evidence", "command": "pyauto-heart tick", + "headline": ("STALE — Heart reports an evidence gap but named no " + "reason. Re-tick and re-assess."), + "detail": ("Nothing is known-bad. STALE means evidence is missing " + "or expired, so the remedy is a fresh run of the check, " + "not a code fix."), + "checkpoint": CHECKPOINT} + else: + cap = t["capability"] + cmd = t["refresh"] + detail = ("Maps to the " + cap + " capability. Nothing is known-bad: " + "STALE means the evidence is missing or expired, so the remedy " + "is a fresh RUN of that check, never a code fix. ") + if cmd is None: + detail += ("Heart exposes no refresh entry point for " + cap + " — " + "re-run that check at its own source, then re-assess. ") + elif cap == "validate": + detail += ("This is the hard readiness gate (Stages 0-3: TestPyPI " + "rehearsal + wheel integration -> Heart ingests -> " + "re-judge), DELEGATED to the release conductor, which owns " + "the MCP GitHub boundary. ") + detail += ("A release still requires GREEN; the dev-ship gate " + "(AUTONOMY.md leg 4) treats STALE as passing, because an " + "evidence gap is organism-scope, not branch-scope.") + rec = {"action": "refresh-evidence", "command": cmd, + "headline": "STALE — refresh the expired evidence: " + t["reason"], + "detail": detail, "checkpoint": CHECKPOINT} elif gaps or verdict == "yellow": # Pure baseline gaps: the classic first-run YELLOW. The one leg that closes # the release-validation gap and makes GREEN reachable is release validate. @@ -291,6 +371,7 @@ print(json.dumps({ "counts": { "blockers": len(blockers), "warnings_real": len(warn_real), + "evidence_gaps": len(evidence), "expected_gaps": len(gaps), "advisory": len(advisory), }, @@ -306,6 +387,10 @@ _exit_code_for() { green) return 0 ;; yellow) return 2 ;; red) return 3 ;; + # 6, not the free slot 1: bash itself exits 1 on a generic failure, and STALE + # PASSES the dev-ship gate — a crash must never be readable as a pass. See + # the exit-code table in this file's header. + stale) return 6 ;; *) return 4 ;; esac } @@ -325,6 +410,7 @@ head += ") --" print(head) print(" " + str(c["blockers"]) + " blocker(s) · " + str(c["warnings_real"]) + " real warning(s) · " + + str(c.get("evidence_gaps", 0)) + " evidence gap(s) · " + str(c["expected_gaps"]) + " expected first-run gap(s) · " + str(c["advisory"]) + " advisory") print() @@ -335,14 +421,17 @@ def show(title, kinds): return print(title) for it in rows: - mark = "✗" if it["severity"] == "red" else "!" + # ? is the freshness glyph: unlike ✗/! it asserts nothing about the + # evidence, only that it is missing or expired. + mark = {"red": "✗", "stale": "?"}.get(it["severity"], "!") line = " " + mark + " [" + it["capability"] + "] " + it["reason"] - if it.get("fix"): - line += " -> " + it["fix"] + if it.get("fix") or it.get("refresh"): + line += " -> " + (it.get("fix") or it.get("refresh")) print(line) print() show("Real problems (act on — these block green):", {"real-problem"}) +show("Evidence gaps (re-run the named check — nothing is known-bad):", {"evidence-gap"}) show("Expected first-run gaps (standing YELLOW — accept, not action items):", {"baseline-gap"}) show("Advisory (monitoring only — does not gate readiness):", {"advisory"}) diff --git a/tests/test_health_conductor.py b/tests/test_health_conductor.py new file mode 100644 index 0000000..5237764 --- /dev/null +++ b/tests/test_health_conductor.py @@ -0,0 +1,278 @@ +"""Contract tests for the health conductor's verdict footing — in particular +Heart's STALE freshness tier (PyAutoBrain#198). + +Hermetic: every test stubs `pyauto-heart` on PATH with a script that echoes a +fabricated readiness payload, so the conductor is exercised end-to-end (through +the vitals faculty, exactly as in production) without touching real Heart state. +`PYAUTO_HEART` points at an empty directory so the optional capability manifest +is absent and triage reasons purely by signal category. + +The bug these pin: the conductor walked only `red_reasons` and `yellow_reasons`, +so a STALE-only verdict produced zero triage items, the UNKNOWN recommendation, +and exit 4 — indistinguishable from "Heart unreachable" for a machine caller, +even though AUTONOMY.md leg 4 treats STALE as *passing* the dev-ship gate. +""" + +import json +import os +import subprocess +from pathlib import Path + +BRAIN_HOME = Path(__file__).resolve().parents[1] +BRAIN = BRAIN_HOME / "bin" / "pyauto-brain" +HEALTH_DIR = BRAIN_HOME / "agents" / "conductors" / "health" +HEALTH_SH = (HEALTH_DIR / "health.sh").read_text() +HEALTH_DOC = (HEALTH_DIR / "AGENTS.md").read_text() + +# Exit codes are a machine contract: 6 is STALE, and it is deliberately NOT the +# free slot 1 (bash's own generic failure), because STALE passes the ship gate. +EXIT_GREEN, EXIT_YELLOW, EXIT_RED, EXIT_UNKNOWN, EXIT_USAGE, EXIT_STALE = 0, 2, 3, 4, 5, 6 + +STALE_ONLY = { + "verdict": "stale", + "score": 75, + "ts": "2026-08-05T10:00:00+00:00", + "red_reasons": [], + "yellow_reasons": [], + "stale_reasons": [ + "no release validation for current source", + "install verification not run", + "PyAutoLens: status unknown", + ], +} + + +def _run(tmp_path, readiness, *args): + """Run the health conductor against a stubbed Heart returning ``readiness``.""" + stub_dir = tmp_path / "bin" + stub_dir.mkdir(exist_ok=True) + payload = tmp_path / "readiness.json" + payload.write_text(json.dumps(readiness)) + heart = stub_dir / "pyauto-heart" + heart.write_text( + "#!/usr/bin/env bash\n" + 'if [[ "${1:-}" == "readiness" ]]; then cat "$STUB_READINESS"; fi\n' + "exit 0\n" + ) + heart.chmod(0o755) + empty_heart_repo = tmp_path / "no_heart_checkout" + empty_heart_repo.mkdir(exist_ok=True) + + env = dict(os.environ) + env["PATH"] = f"{stub_dir}{os.pathsep}{env['PATH']}" + env["STUB_READINESS"] = str(payload) + env["PYAUTO_HEART"] = str(empty_heart_repo) + return subprocess.run( + [str(BRAIN), "health", *args], capture_output=True, text=True, env=env, + ) + + +def _triage(tmp_path, readiness): + result = _run(tmp_path, readiness, "triage", "--json") + return json.loads(result.stdout), result.returncode + + +# -------------------------------------------------------------------------- +# 1. the classifier walks stale_reasons +# -------------------------------------------------------------------------- + +def test_stale_reasons_become_triage_items(tmp_path): + t, _ = _triage(tmp_path, STALE_ONLY) + reasons = [it["reason"] for it in t["items"]] + assert sorted(reasons) == sorted(STALE_ONLY["stale_reasons"]) + assert {it["severity"] for it in t["items"]} == {"stale"} + + +def test_stale_items_are_evidence_gaps_not_accepted_baseline_gaps(tmp_path): + """A stale reason is an *action item* (re-run the check), so it must not land + in the bucket the renderer labels "accept, not action items".""" + t, _ = _triage(tmp_path, STALE_ONLY) + assert {it["kind"] for it in t["items"]} == {"evidence-gap"} + assert t["counts"]["expected_gaps"] == 0 + assert all(it["blocks_green"] for it in t["items"]) + + +def test_stale_counts_are_no_longer_zero(tmp_path): + """The reported symptom: counts read 0 blockers / 0 warnings / 0 gaps.""" + t, _ = _triage(tmp_path, STALE_ONLY) + c = t["counts"] + assert c["evidence_gaps"] == len(STALE_ONLY["stale_reasons"]) + assert (c["blockers"], c["warnings_real"], c["advisory"]) == (0, 0, 0) + + +# -------------------------------------------------------------------------- +# 2. the recommendation chain has a STALE branch +# -------------------------------------------------------------------------- + +def test_stale_recommendation_is_refresh_not_unknown(tmp_path): + t, _ = _triage(tmp_path, STALE_ONLY) + rec = t["recommendation"] + assert rec["action"] == "refresh-evidence" + assert rec["headline"].startswith("STALE") + assert "UNKNOWN" not in rec["headline"] + assert "could not obtain a verdict" not in rec["detail"] + + +def test_stale_detail_states_the_tier_semantics(tmp_path): + """The remedy is re-running the check, never fixing code — and a release + still needs GREEN even though the dev-ship gate treats STALE as passing.""" + t, _ = _triage(tmp_path, STALE_ONLY) + detail = t["recommendation"]["detail"].lower() + assert "missing or expired" in detail + assert "never a code fix" in detail + assert "requires green" in detail + assert "leg 4" in detail + + +def test_stale_prefers_the_release_validation_gap(tmp_path): + """Refreshing the hard readiness gate is the leg that makes GREEN reachable, + so it is recommended ahead of other evidence gaps regardless of input order.""" + t, _ = _triage(tmp_path, STALE_ONLY) + rec = t["recommendation"] + assert "no release validation" in rec["headline"] + assert rec["command"] == "pyauto-brain release validate" + + +def test_refresh_commands_are_never_invented(tmp_path): + """A capability Heart exposes no refresh entry point for yields command None + and says so in prose — the same discipline as the `pyauto-heart fix` topics.""" + t, _ = _triage(tmp_path, { + "verdict": "stale", "score": 90, + "red_reasons": [], "yellow_reasons": [], + "stale_reasons": ["test run stale (12d old)"], + }) + rec = t["recommendation"] + assert rec["command"] is None + assert "no refresh entry point" in rec["detail"] + assert t["items"][0]["capability"] == "test_run" + + +def test_stale_verdict_with_no_named_reason_still_recommends_a_refresh(tmp_path): + """Defensive: a verdict of stale with an empty reason list must not fall + through to UNKNOWN.""" + t, code = _triage(tmp_path, { + "verdict": "stale", "score": 80, + "red_reasons": [], "yellow_reasons": [], "stale_reasons": [], + }) + assert t["recommendation"]["action"] == "refresh-evidence" + assert code == EXIT_STALE + + +# -------------------------------------------------------------------------- +# 3. the exit code — distinguishable from unknown by a machine caller +# -------------------------------------------------------------------------- + +def test_stale_exits_six(tmp_path): + _, code = _triage(tmp_path, STALE_ONLY) + assert code == EXIT_STALE + + +def test_every_verdict_maps_to_a_distinct_exit_code(tmp_path): + """The point of the fix: "Heart says STALE" and "Heart unreachable" must not + collapse onto the same code.""" + cases = { + EXIT_GREEN: {"verdict": "green", "red_reasons": [], "yellow_reasons": [], + "stale_reasons": []}, + EXIT_YELLOW: {"verdict": "yellow", "red_reasons": [], + "yellow_reasons": ["PyAutoFit: uncommitted changes"], + "stale_reasons": []}, + EXIT_RED: {"verdict": "red", "red_reasons": ["PyAutoFit: CI failing on main"], + "yellow_reasons": [], "stale_reasons": []}, + EXIT_UNKNOWN: {}, + EXIT_STALE: STALE_ONLY, + } + observed = {expected: _triage(tmp_path, payload)[1] + for expected, payload in cases.items()} + assert observed == {code: code for code in cases} + + +def test_usage_error_stays_five(tmp_path): + result = _run(tmp_path, STALE_ONLY, "not-a-subcommand") + assert result.returncode == EXIT_USAGE + + +def test_stale_exit_code_is_not_the_shell_generic_failure(): + """1 is bash's own generic failure (a missing `_common.sh`, a failed + `readlink`). STALE passes the dev-ship gate, so a crash must never be + readable as a passing verdict — hence 6, not the free slot 1.""" + assert EXIT_STALE != 1 + assert "stale) return 6 ;;" in HEALTH_SH + assert "return 1" not in HEALTH_SH.split("_exit_code_for() {")[1].split("}")[0] + + +def test_header_exit_code_table_documents_stale(): + """The documented table and `_exit_code_for` must not drift apart.""" + header = HEALTH_SH.split("set -uo pipefail")[0] + assert "6 stale" in header + for verdict, code in (("green", 0), ("yellow", 2), ("red", 3), ("unknown", 4)): + assert f"{code} {verdict}" in header, verdict + # the rationale for 6 travels with the table, not just the commit message + assert "PASSING" in header or "passing" in header + + +def test_agents_doc_documents_the_stale_tier(): + """The conductor's own AGENTS.md is the second copy of the exit-code table + and the triage taxonomy — both must carry the tier, or a reader is told the + pre-fix story.""" + assert "`6` stale" in HEALTH_DOC + assert "**STALE**" in HEALTH_DOC + assert "Evidence gaps" in HEALTH_DOC + assert "four kinds" in HEALTH_DOC + + +# -------------------------------------------------------------------------- +# 4. regression guards — the other tiers are untouched, and an older Heart works +# -------------------------------------------------------------------------- + +def test_heart_without_the_stale_tier_behaves_as_before(tmp_path): + """A verdict from a Heart predating the tier has no `stale_reasons` key at + all: the conductor must degrade to the old behaviour, not to UNKNOWN.""" + t, code = _triage(tmp_path, { + "verdict": "yellow", "score": 70, "red_reasons": [], + "yellow_reasons": ["no release validation for current source"], + }) + assert code == EXIT_YELLOW + assert t["recommendation"]["action"] == "release-validate" + assert t["counts"]["evidence_gaps"] == 0 + assert t["counts"]["expected_gaps"] == 1 + + +def test_red_still_dominates_a_stale_reason(tmp_path): + """Heart's precedence is red > yellow > stale; the conductor must adopt it, + never recommend a refresh while a blocker is open.""" + t, code = _triage(tmp_path, { + "verdict": "red", "score": 40, + "red_reasons": ["PyAutoFit: CI failing on main"], + "yellow_reasons": [], + "stale_reasons": ["install verification not run"], + }) + assert code == EXIT_RED + assert t["recommendation"]["action"] == "resolve-blockers" + assert t["counts"]["blockers"] == 1 + + +def test_yellow_real_warning_still_wins_over_a_stale_reason(tmp_path): + t, code = _triage(tmp_path, { + "verdict": "yellow", "score": 60, "red_reasons": [], + "yellow_reasons": ["PyAutoLens: uncommitted changes"], + "stale_reasons": ["install verification not run"], + }) + assert code == EXIT_YELLOW + assert t["recommendation"]["action"] == "resolve-warning" + + +# -------------------------------------------------------------------------- +# 5. the human render +# -------------------------------------------------------------------------- + +def test_human_render_shows_the_evidence_gap_section(tmp_path): + result = _run(tmp_path, STALE_ONLY, "triage") + assert result.returncode == EXIT_STALE + out = result.stdout + assert "adopted verdict: STALE" in out + assert "Evidence gaps (re-run the named check" in out + assert "3 evidence gap(s)" in out + # the freshness glyph asserts nothing about the evidence, unlike ✗ / ! + assert "? [validate] no release validation for current source" in out + assert "pyauto-brain release validate" in out + assert "UNKNOWN" not in out