From 875330ff79ab62d5175d777f6c03be372ceb1422 Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 10 Aug 2026 19:36:24 +0000 Subject: [PATCH 1/2] =?UTF-8?q?feat:=20intake=20reconcile=20--repo=20?= =?UTF-8?q?=E2=80=94=20read=20the=20target=20repo=20for=20staleness=20evid?= =?UTF-8?q?ence?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Leg 3 of the draft-staleness work. Legs 1 and 2 shipped Mind-local signals: the lifecycle.py gate keys, and the reconcile re-rank that cut the flag rate from 96/148 to 29/135. Both are structurally blind to two of the five findings the 2026-08-09 sweep confirmed — one whose evidence sat in a sibling PROMPT rather than a completion record, and one whose fix shipped upstream with no record written at all. Re-ranking cannot reach those; only reading the target repo can. This adds that read. `intake reconcile --repo ` resolves the target through the body map plus the sizing faculty's existing aliases, makes a cached shallow clone, greps for the identifiers each prompt names, and reports them as file:line evidence in a `needs-review` band. The resolved sha is printed so a verdict is re-checkable. Upstream hits deliberately do NOT feed `overlap_score`. The reason is a measured trap: test_mode_bypass_ordered_assertion_ties names five identifiers, all five ARE on PyAutoFit main, and the prompt is NOT shipped — main's catch wraps only the likelihood call while the line that actually raises sits before the `try`. Scoring presence as evidence would rank that prompt `high` and make the one mis-grade this tool must never make. Verified end-to-end against PyAutoFit fbe9f45d: it lands in `needs-review` with overlap_score 0.0, surfaced but never called shipped. Two noise classes were found on the first real run and filtered: Python builtins (`TypeError`, 37 files) and repo names (`autofit_workspace`, 26). Filtering by upstream file-spread instead was tried and rejected — the counts do not separate, since `instance_from_vector` is a real signal at 22 files. This is the first network access in PyAutoBrain, so it is strictly opt-in: without --repo no clone, socket or subprocess runs, and a test detonates on any attempt. AUTONOMY.md records the surface and that widening it is a new decision. Multi-repo targets (`workspaces`, `priors`, …) exit 5 naming the real candidates rather than guessing one — they are the largest buckets in draft/, so a guess would be confidently wrong at scale. Tests: 8 new, all hermetic via an injected source_reader — nothing under tests/ clones. 330 passed. Plain --depth 1 over the treeless clone the parent prompt suggested: grepping source refetches every blob on demand. Closes #223 Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01PY39MRi1oPbxhhr6tDToQU --- AUTONOMY.md | 9 + agents/conductors/intake/AGENTS.md | 25 +++ agents/conductors/intake/_intake.py | 289 ++++++++++++++++++++++++- agents/conductors/intake/intake.sh | 7 + skills/intake/intake.md | 8 + tests/test_intake_reconcile_ranking.py | 141 ++++++++++++ 6 files changed, 467 insertions(+), 12 deletions(-) diff --git a/AUTONOMY.md b/AUTONOMY.md index 79e3b3b..1498faa 100644 --- a/AUTONOMY.md +++ b/AUTONOMY.md @@ -300,6 +300,15 @@ tier), never by weakening leg 4. while Heart is RED. - **Never rewrite history** (`AGENTS.md` rules apply verbatim to autonomous runs). +- **The Brain reads no network except one opt-in flag.** Every conductor and + faculty is stdlib-only and offline. The sole exception is + `intake reconcile --repo ` (PyAutoBrain#223), which makes a cached + shallow clone of the named repo to check whether identifiers a prompt names + already exist upstream. It is **read-only, opt-in and non-default**: without + the flag no clone, socket or subprocess is used, and a test detonates on any + attempt. It ranks prompts for human review in a `needs-review` band and never + emits a shipped verdict, so it retires nothing on its own. Any *further* + network surface in the Brain is a new decision, not covered by this line. - The `Autonomy:` header is a model's own estimate. The caps, the explicit `--auto` launch, and the calibration log are what make consuming it defensible — none of the three is optional. diff --git a/agents/conductors/intake/AGENTS.md b/agents/conductors/intake/AGENTS.md index 66e85d1..d34e9b4 100644 --- a/agents/conductors/intake/AGENTS.md +++ b/agents/conductors/intake/AGENTS.md @@ -86,6 +86,7 @@ schema — light structure over free-form prose. | **dashboard** | `intake dashboard` | render the census as the Mind **backlog** page; `--apply` writes `PyAutoMind/dashboard.md` | | **formalise** | `intake formalise [prefix]` | retroactively header the prompts census flags — derive the missing fields, insert in place, prose untouched; `--apply` writes | | **reconcile** | `intake reconcile [prefix]` | rank backlog prompts that look already-shipped (vs the `complete/` records / `active/`); always read-only — retiring stays human | +| **reconcile --repo** | `intake reconcile --repo [prefix]` | **also** read the target repo's source for identifiers the prompts name — the one signal that sees a prompt with no Mind-side trace. Opt-in; the default path is offline | Census/dashboard are the Mind *backlog* view — deliberately distinct from Heart's `/health status` health view (see "must never do"). The prompt-taxonomy @@ -106,6 +107,30 @@ basenames, and hand-set Status values, then ranks suspects (high/medium/low) with the evidence shown. The final verification — the target repo's git log / merged PRs — and the retirement itself stay human. +`--repo ` adds the **upstream leg**: identifiers a prompt names that +already exist in the target repo's source, cited as `file:line`, in their own +weaker `needs-review` band. It is the only signal that reaches a prompt with no +Mind-side trace at all — the 2026-08-09 sweep confirmed two such findings, one +whose evidence sat in a sibling *prompt* and one whose fix shipped with no +completion record ever written. + +It never produces a shipped verdict, and upstream hits are scored on their own +key so they cannot inflate a Mind-local band. The reason is a measured trap: +`test_mode_bypass_ordered_assertion_ties.md` names five identifiers, all five +are on PyAutoFit `main`, and the prompt is **not** shipped — the upstream catch +wraps only the likelihood call while the raising line sits before the `try`. +Presence of a name is not presence of the fix. + +**This is the only network access in PyAutoBrain.** Every other conductor and +faculty is stdlib-only and offline, and the default `reconcile` path stays that +way — a test detonates on any socket or subprocess use when `--repo` is absent. +A target that is not one repo (`workspaces`, `health_fixes`, `priors`, +`graphical_ep` — topic clusters, and among the largest buckets in `draft/`) is +**refused** with exit `5` naming the real candidates, never silently guessed. +Clones are cached shallow (`--depth 1`) under `$PYAUTO_BRAIN_CACHE` +(default `~/.pyauto-brain/upstream`), and the resolved sha is printed so a +verdict is re-checkable. + ## Machine sources (one staging surface) Conception input increasingly arrives from the organism itself — research diff --git a/agents/conductors/intake/_intake.py b/agents/conductors/intake/_intake.py index a992635..bb602b8 100755 --- a/agents/conductors/intake/_intake.py +++ b/agents/conductors/intake/_intake.py @@ -26,6 +26,7 @@ import datetime as _dt import json import math +import os import re import sys from pathlib import Path @@ -37,7 +38,7 @@ from _sizing import ( # noqa: E402 WORK_TYPES, LIBRARY_REPOS, WORKSPACE_REPOS, ORGANISM_REPOS, KNOWN_REPOS, RISK_KEYWORDS, AMBIGUITY_KEYWORDS, normalise_repo, estimate_difficulty, _hits, - policy as _sizing_policy, + policy as _sizing_policy, BODY_MAP_PATH, ) # --- work-type classification ------------------------------------------------- @@ -656,7 +657,202 @@ def _idents(text: str) -> set: return set(_IDENT_RE.findall(text)) -def reconcile(mind: Path, prefix: str = "") -> dict: +# --- the upstream read (--repo) ------------------------------------------------- +# Leg 3 of the staleness work (PyAutoBrain#223). Everything above this line is +# Mind-local: it cross-references `draft/` against `complete/` and `active/`. +# That is structurally blind to two of the five findings the 2026-08-09 sweep +# confirmed — one whose evidence sat in a sibling PROMPT rather than a record, +# and one whose fix shipped upstream with NO record written at all. Re-ranking +# cannot reach them; only reading the target repo can. +# +# THIS IS THE ONLY NETWORK ACCESS IN PyAutoBrain. Every other conductor and +# faculty is stdlib-only and offline, and the default `reconcile` path stays +# that way — `--repo` is strictly opt-in, and `test_default_path_is_offline` +# pins it. The upstream read goes through the `source_reader` seam so the +# hermetic tests never clone anything. +# +# What it must NEVER do is call a prompt shipped. `test_mode_bypass_ordered_ +# assertion_ties.md` names five identifiers and ALL FIVE are on PyAutoFit main, +# yet the prompt is confirmed not shipped: main catches `exc.FitException` in +# the TEST_MODE bypass — which looks exactly like the requested fix — but the +# catch wraps only the likelihood call, while `model.instance_from_vector` +# (where `check_assertions` actually raises) sits on the line BEFORE the `try`. +# Presence of a name is not presence of the fix. So this leg contributes +# evidence and a `needs-review` band, never a verdict. +_W_UPSTREAM = 1.5 # per upstream identifier beyond the first. Deliberately + # below _W_SHIPPED (7.0): an upstream hit must never on + # its own carry a prompt into the top band, because the + # trap above would ride it there. +_UPSTREAM_MIN_IDENTS = 2 # one shared name is a coincidence, not a signal + + +def _upstream_noise() -> set: + """Identifiers whose presence upstream says nothing about a prompt. + + Two measured noise classes, from the first run against PyAutoFit: + + * **Python builtins** — `TypeError` is in 37 files of PyAutoFit. A prompt + mentioning it has not thereby been shipped. + * **Repo names** — `autofit_workspace` (26 files), `autolens_workspace`. + Every repo names its siblings; that is vocabulary, not evidence. + + Filtering by upstream file-spread instead was tried and rejected: the counts + do not separate. `instance_from_vector` (22 files) is a REAL signal and sits + right below `autofit_workspace` (26) which is noise, so any threshold that + drops the noise also drops one of the trap's own identifiers. + """ + import builtins + + return set(dir(builtins)) | set(KNOWN_REPOS) + + +def _body_map_slugs() -> dict: + """normalised target -> `owner/repo`, from the Mind's body map. + + repos.yaml is the single source of repo identity, and `normalise_repo` + already folds `PyAutoArray`/`pyautoarray`/`autoarray` together — reuse both + rather than adding a third mapping. + """ + import yaml + + data = yaml.safe_load(BODY_MAP_PATH.read_text()) + out = {} + for name, spec in data["repos"].items(): + slug = spec.get("github", "") + if slug: + out[normalise_repo(name)] = slug + return out + + +def _target_candidates(mind: Path, target: str) -> list: + """Repos actually referenced by the prompts filed under `draft/**//`. + + Used only to make the refusal below useful: `--repo priors` should say which + repos those six prompts are about, not just "no". + """ + found = set() + for wt in WORK_TYPES: + folder = mind / "draft" / wt / target + if not folder.is_dir(): + continue + for f in folder.rglob("*.md"): + text = f.read_text(encoding="utf-8", errors="replace") + for m in re.findall(r"@([A-Za-z_][A-Za-z0-9_]*)", text): + key = normalise_repo(m) + if key in KNOWN_REPOS: + found.add(key) + return sorted(found) + + +def resolve_repo(mind: Path, target: str) -> tuple: + """`target` -> (`owner/repo`, ""), or ("", ) if it is not one repo. + + The second folder of a prompt path is a target *or domain*: `autoarray` is a + repo, but `workspaces`, `health_fixes`, `priors` and `graphical_ep` are topic + clusters spanning several. Those are among the LARGEST buckets in `draft/` + (`workspaces` alone is 23 prompts across four work-types), so guessing one + repo for them would produce confident nonsense over the biggest part of the + backlog. Refuse instead, and name the real candidates. + """ + key = normalise_repo(target) + slugs = _body_map_slugs() + if key in slugs: + return slugs[key], "" + cands = _target_candidates(mind, target) + if cands: + hint = (" the prompts filed under it reference: " + ", ".join(cands) + + "\n re-run --repo with one of those.") + else: + hint = (" no @RepoName references found in the prompts filed under it" + "\n re-run --repo with a repo name from the body map.") + return "", (f"--repo {target!r} is not a single repository in the body map " + f"(PyAutoMind/repos.yaml).\n{hint}") + + +def _clone_upstream(slug: str, cache: Path) -> tuple: + """Cached shallow clone of `slug`'s default branch -> (path, sha). + + Plain `--depth 1`, NOT the `--filter=blob:none` treeless clone the parent + prompt suggested: this leg greps the source, and a treeless clone refetches + every blob on demand to answer that — a false economy. GIT_LFS_SKIP_SMUDGE + keeps an LFS-using repo from aborting at the smudge filter. + """ + import subprocess + + dest = cache / slug.replace("/", "__") + env = {**os.environ, "GIT_LFS_SKIP_SMUDGE": "1", "GIT_TERMINAL_PROMPT": "0"} + + def _git(*args, cwd=None): + return subprocess.run(["git", *args], cwd=cwd, env=env, + capture_output=True, text=True, timeout=600) + + if (dest / ".git").is_dir(): + _git("fetch", "--depth", "1", "origin", cwd=dest) + _git("reset", "--hard", "FETCH_HEAD", cwd=dest) + else: + dest.parent.mkdir(parents=True, exist_ok=True) + r = _git("clone", "--depth", "1", + f"https://github.com/{slug}", str(dest)) + if r.returncode != 0: + raise RuntimeError(f"clone of {slug} failed: {r.stderr.strip()}") + sha = _git("rev-parse", "HEAD", cwd=dest).stdout.strip() + return dest, sha + + +def _grep_source(root: Path, idents: set) -> dict: + """ident -> [':', …] over the source files under `root`. + + One pass over the tree scoring every identifier at once: the alternative, + one grep per identifier per prompt, is O(prompts x idents) walks of the + same checkout. + """ + if not idents: + return {} + pat = re.compile(r"\b(" + "|".join(re.escape(i) for i in sorted(idents)) + + r")\b") + hits: dict = {} + for f in root.rglob("*"): + if not f.is_file() or ".git" in f.parts: + continue + if f.suffix not in (".py", ".pyi", ".sh", ".yaml", ".yml", ".cfg", + ".toml", ".rst", ".md", ".ipynb"): + continue + try: + text = f.read_text(encoding="utf-8", errors="replace") + except OSError: + continue + if not pat.search(text): + continue + rel = f.relative_to(root) + for n, line in enumerate(text.splitlines(), 1): + for m in pat.finditer(line): + hits.setdefault(m.group(1), []) + if len(hits[m.group(1)]) < 3: # 3 lines is enough to judge + hits[m.group(1)].append(f"{rel}:{n}") + return hits + + +def upstream_reader(mind: Path, target: str, cache: Path = None): + """Build the `source_reader` seam for `reconcile(repo=…)`. + + Returns `(reader, sha, slug, err)`. `reader(idents) -> {ident: [file:line]}`. + Kept separate from `reconcile` so the tests can inject a fake tree and stay + hermetic — nothing under `tests/` ever clones. + """ + slug, err = resolve_repo(mind, target) + if err: + return None, "", "", err + cache = cache or Path(os.environ.get( + "PYAUTO_BRAIN_CACHE", Path.home() / ".pyauto-brain" / "upstream")) + try: + root, sha = _clone_upstream(slug, cache) + except Exception as exc: # network/git failure + return None, "", slug, f"could not read {slug}: {exc}" + return (lambda idents: _grep_source(root, idents)), sha, slug, "" + + +def reconcile(mind: Path, prefix: str = "", source_reader=None, + upstream_meta: dict = None) -> dict: """Rank backlog prompts that look already-shipped, for a human to retire. Mind-local signals per prompt: a completion-record line referencing its @@ -664,6 +860,12 @@ def reconcile(mind: Path, prefix: str = "") -> dict: or the `complete/` archive, token overlap with a completed task's header / archive record, and a hand-set Status the formalise pass deliberately preserved. Never writes anything. + + `source_reader` is the optional upstream leg (`--repo`): a callable taking + the identifiers a prompt names and returning `{ident: ['file:line', …]}` + from the target repo's source. It ADDS evidence and can raise a prompt into + the `needs-review` band; it never produces a shipped verdict, and it is + never consulted unless the caller passes it. Default: offline. """ c = census(mind) comp_dir = mind / "complete" @@ -703,6 +905,7 @@ def reconcile(mind: Path, prefix: str = "") -> dict: active = mind / "active" issued_names = ({p.name for p in active.glob("*.md")} if active.is_dir() else set()) + _noise = _upstream_noise() if source_reader is not None else set() suspects = [] for r in c["records"]: @@ -806,43 +1009,91 @@ def reconcile(mind: Path, prefix: str = "") -> dict: f"record '{top}' names {n} of this prompt's " f"identifiers: {', '.join(sorted(hits[top])[:5])}")) + # 4. The upstream leg (--repo): identifiers this prompt names that are + # ALREADY PRESENT in the target repo's source. This is the only leg + # that can see a prompt with no Mind-side trace at all. + # + # It deliberately does NOT add to `score`. Presence of a name is not + # presence of the fix — test_mode_bypass_ordered_assertion_ties names + # five identifiers, all five are upstream, and the prompt is NOT + # shipped. Letting upstream hits feed `score` would carry exactly + # that prompt into the `high` band and make the one mis-grade this + # tool must never make. So upstream evidence gets its own weaker + # band and its own ordering key, and can never inflate a Mind-local + # verdict. + upstream_score = 0.0 + if source_reader is not None: + all_ids = {i for i in _idents(prompt_text) + if normalise_repo(i) not in _noise and i not in _noise} + up = source_reader(all_ids) if all_ids else {} + if len(up) >= _UPSTREAM_MIN_IDENTS: + upstream_score = _W_UPSTREAM * (len(up) - 1) + shown = sorted(up)[:5] + findings.append(( + "upstream-identifier-present", + f"{len(up)} of this prompt's identifiers already exist " + f"upstream: " + "; ".join( + f"{i} ({up[i][0]})" for i in shown))) + # `Status:` alone is not evidence — it fired on every hand-set draft. Kept # as context on prompts something else already flagged, never as a reason. - if score > 0 and r["status"] not in ("-", "formalised"): + if (score > 0 or upstream_score > 0) and r["status"] not in ("-", "formalised"): findings.append(("stale-status", f"Status: {r['status']} — hand-set; verify against " "shipped state")) - if score >= _SUSPECT_THRESHOLD: - conf = "high" if score >= _HIGH_THRESHOLD else "medium" + if score >= _SUSPECT_THRESHOLD or upstream_score > 0: + if score >= _HIGH_THRESHOLD: + conf = "high" + elif score >= _SUSPECT_THRESHOLD: + conf = "medium" + else: + # Upstream evidence only — the prompt has no Mind-side signal. + # This is the band leg 3 exists to produce. + conf = "needs-review" suspects.append({ "path": path, "title": r["title"], "confidence": conf, + "upstream_score": round(upstream_score, 2), "overlap_score": round(score, 2), "findings": [{"kind": k, "evidence": e} for k, e in findings], }) - order = {"high": 0, "medium": 1, "low": 2} - suspects.sort(key=lambda s: (order[s["confidence"]], - -s["overlap_score"], s["path"])) + # `needs-review` sorts BELOW the Mind-local bands: an upstream name-match is + # weaker evidence than a record saying the work shipped, and the ordering + # should say so. + order = {"high": 0, "medium": 1, "low": 2, "needs-review": 3} + suspects.sort(key=lambda s: (order[s["confidence"]], -s["overlap_score"], + -s.get("upstream_score", 0.0), s["path"])) return {"generated": _dt.date.today().isoformat(), "scanned": c["total"], - "suspects": suspects} + "upstream": upstream_meta or {}, "suspects": suspects} def emit_reconcile(res: dict): print(f"== Intake reconcile: {len(res['suspects'])} suspect(s) of " f"{res['scanned']} scanned ==") + up = res.get("upstream") or {} + if up: + # The sha is what makes a verdict re-checkable: "these names were on + # main at THIS commit" is a claim someone can go and re-run. + print(f" upstream: {up['slug']} @ {up['sha'][:12]} " + f"(read {up['when']})") if not res["suspects"]: print(" backlog reconciles clean against the complete/ records " "and active/.") for s in res["suspects"]: - print(f"[{s['confidence']:>6}] {s['path']}") + print(f"[{s['confidence']:>12}] {s['path']}") for f in s["findings"]: ev = f["evidence"] if len(ev) > 160: ev = ev[:157] + "…" - print(f" {f['kind']}: {ev}") + print(f" {f['kind']}: {ev}") print("\nRetiring a prompt stays human: verify against the target repo's " "git log / merged\nPRs, then retire it to the complete/ archive by hand.") + if up: + print("`needs-review` means the prompt NAMES things that exist upstream " + "— NOT that it\nshipped. A fix can land next to the name without " + "being the fix the prompt asks for;\nread the cited lines before " + "retiring anything.") # --- ideas.md scanning -------------------------------------------------------- @@ -941,6 +1192,11 @@ def main(argv=None): "already-shipped (always read-only)") rc.add_argument("prefix", nargs="?", default="", help="only reconcile prompts under this path prefix") + rc.add_argument("--repo", default="", + help="ALSO read this target repo's source for identifiers " + "the prompts name (the only leg that sees prompts with " + "no Mind-side trace). Opt-in: the default path makes no " + "network access. Ranks for review; never says shipped.") a = ap.parse_args(argv) mind = Path(a.mind) @@ -954,7 +1210,16 @@ def main(argv=None): if a.apply: print("intake reconcile is read-only — retiring prompts stays " "human (--apply ignored).", file=sys.stderr) - res = reconcile(mind, prefix=a.prefix) + reader, meta = None, None + if a.repo: + reader, sha, slug, err = upstream_reader(mind, a.repo) + if err: + print(f"intake reconcile: {err}", file=sys.stderr) + return 5 + meta = {"slug": slug, "sha": sha, + "when": _dt.date.today().isoformat()} + res = reconcile(mind, prefix=a.prefix, source_reader=reader, + upstream_meta=meta) print(json.dumps(res, indent=2)) if a.as_json else emit_reconcile(res) return 0 diff --git a/agents/conductors/intake/intake.sh b/agents/conductors/intake/intake.sh index ce4d688..2c7755c 100755 --- a/agents/conductors/intake/intake.sh +++ b/agents/conductors/intake/intake.sh @@ -27,6 +27,13 @@ # intake reconcile [prefix] rank backlog prompts that look already-shipped # (vs the complete/ records / active/); always read-only — # retiring a prompt stays human +# --repo ALSO read 's source for identifiers the +# prompts name — the only signal that sees a prompt +# with no Mind-side trace. Opt-in: without it no +# network access happens. Ranks for review in a +# `needs-review` band; never says shipped. A target +# that is not one repo (workspaces, priors, …) is +# refused, not guessed. # # Flags (place before the subcommand; both default OFF): # --apply write the formal prompt file(s) / dashboard.md; else dry-run only diff --git a/skills/intake/intake.md b/skills/intake/intake.md index dc16fb2..b90ad9d 100644 --- a/skills/intake/intake.md +++ b/skills/intake/intake.md @@ -44,6 +44,14 @@ dev workflow (issue, branch, plan). Do not bypass the Brain. hand-set `Status:` is a signal, never proof). Always read-only: verify each suspect against the target repo's git log / merged PRs, then retire it to the `complete/` archive by hand (it is already done). +- `bin/pyauto-brain intake reconcile --repo [prefix]` — **also** reads + ``'s source for identifiers the prompts name, citing `file:line` in a + weaker `needs-review` band. This is the only signal that reaches a prompt with + no Mind-side trace, and it is the Brain's only network access — opt-in, cached + shallow clone, still read-only. It never says *shipped*: a name can exist + upstream without the prompt's fix existing, so read the cited lines before + retiring anything. A target that is not one repo (`workspaces`, `priors`, …) + is refused with the real candidates named, never guessed. ## Boundary diff --git a/tests/test_intake_reconcile_ranking.py b/tests/test_intake_reconcile_ranking.py index 756244f..88e0ada 100644 --- a/tests/test_intake_reconcile_ranking.py +++ b/tests/test_intake_reconcile_ranking.py @@ -192,3 +192,144 @@ def test_suspects_carry_their_evidence_and_a_band(tmp_path): assert s["confidence"] in ("high", "medium") assert s["findings"] and all(f["kind"] and f["evidence"] for f in s["findings"]) assert any(f["kind"] == "record-says-shipped" for f in s["findings"]) + + +# --------------------------------------------------------------------------- # +# leg 3 — the upstream read (--repo), PyAutoBrain#223 +# +# Everything above is Mind-local. Leg 3 adds the one signal that can see a +# prompt with NO Mind-side trace: identifiers it names that already exist in the +# target repo's source. These tests inject a fake source tree through the +# `source_reader` seam, so this file stays hermetic — nothing here clones. +# --------------------------------------------------------------------------- # +def _reader(table): + """Fake `source_reader`: {ident: [file:line]} for idents in `table`.""" + return lambda idents: {i: table[i] for i in idents if i in table} + + +def test_upstream_presence_never_produces_a_shipped_verdict(tmp_path): + """THE acceptance criterion, in miniature. + + `test_mode_bypass_ordered_assertion_ties.md` names five identifiers and ALL + FIVE are on PyAutoFit main — yet the prompt is confirmed NOT shipped: main + catches `exc.FitException` in the TEST_MODE bypass, which looks exactly like + the requested fix, but the catch wraps only the likelihood call while + `model.instance_from_vector` (where `check_assertions` raises) sits on the + line BEFORE the `try`. + + A matcher that scored presence as shipped would rank this `high` and be + wrong. It must land in the weak band, carrying its evidence. + """ + idents = ["FitException", "check_assertions", "ignore_assertions", + "instance_for_arguments", "instance_from_vector"] + body = "# Ordered assertion ties\n\n" + "\n".join(f"- `{i}`" for i in idents) + root = _mind(tmp_path, {"bug/gearbox/ordered_ties.md": body}, {}) + res = _intake.reconcile( + root, source_reader=_reader({i: [f"src/{i}.py:1"] for i in idents})) + + s = [x for x in res["suspects"] + if x["path"] == "draft/bug/gearbox/ordered_ties.md"] + assert s, "a prompt whose identifiers are all upstream must be surfaced" + assert s[0]["confidence"] == "needs-review" + assert s[0]["confidence"] != "high" + assert any(f["kind"] == "upstream-identifier-present" for f in s[0]["findings"]) + + +def test_upstream_evidence_cannot_inflate_a_mind_local_band(tmp_path): + """The structural defence behind the test above: upstream hits are scored on + their own key and never added to `overlap_score`, so no number of them can + push a prompt up into `high`.""" + idents = [f"`_gearbox_part_{i}`" for i in range(9)] + root = _mind(tmp_path, + {"bug/gearbox/ordered_ties.md": "# Ties\n\n" + "\n".join(idents)}, + {}) + table = {f"_gearbox_part_{i}": [f"src/p{i}.py:1"] for i in range(9)} + res = _intake.reconcile(root, source_reader=_reader(table)) + s = res["suspects"][0] + assert s["confidence"] == "needs-review" + assert s["overlap_score"] == 0.0 # Mind-local score untouched + assert s["upstream_score"] > 0.0 + + +def test_one_upstream_identifier_is_not_a_signal(tmp_path): + """A single shared name is a coincidence — the same bar the Mind-local + identifier leg sets at two.""" + root = _mind(tmp_path, + {"bug/gearbox/ordered_ties.md": "# Ties\n\n`_lone_helper`\n"}, {}) + res = _intake.reconcile( + root, source_reader=_reader({"_lone_helper": ["src/a.py:1"]})) + assert _paths(res) == set() + + +def test_builtins_and_repo_names_are_not_upstream_evidence(tmp_path): + """Measured noise from the first real run: `TypeError` is in 37 files of + PyAutoFit and `autofit_workspace` in 26. Every repo names its siblings and + every codebase raises builtins; neither says a prompt shipped. + + Filtering on upstream file-spread instead was tried and rejected — the + counts do not separate (`instance_from_vector` is a real signal at 22 files, + just under `autofit_workspace` at 26).""" + body = "# Noise only\n\n`TypeError`\n`autofit_workspace`\n`autolens_workspace`\n" + root = _mind(tmp_path, {"bug/gearbox/noise_only.md": body}, {}) + res = _intake.reconcile(root, source_reader=_reader({ + "TypeError": ["src/a.py:1"], + "autofit_workspace": ["README.md:6"], + "autolens_workspace": ["README.md:7"], + })) + assert _paths(res) == set() + + +def test_default_path_makes_no_network_access(tmp_path, monkeypatch): + """PyAutoBrain is otherwise stdlib-only and offline. `--repo` is opt-in, and + the default path must stay provably offline — so detonate on any attempt to + clone or open a socket when no reader was passed.""" + import socket + import subprocess + + def _boom(*a, **k): + raise AssertionError("default reconcile path attempted network access") + + monkeypatch.setattr(subprocess, "run", _boom) + monkeypatch.setattr(socket, "socket", _boom) + root = _mind( + tmp_path, + {"bug/flywheel/sprocket_wobble.md": "# Sprocket wobble\n\nIt wobbles.\n"}, + {"gadget-alignment.md": "## gadget-alignment\n- notes: sprocket_wobble.md " + "shipped to main.\n"}, + ) + assert _intake.reconcile(root)["suspects"] # ran, and stayed offline + + +def test_upstream_mode_still_never_writes(tmp_path): + """The read-only contract holds on the new leg too.""" + root = _mind(tmp_path, + {"bug/gearbox/ordered_ties.md": "# Ties\n\n`_a_helper`\n`_b_helper`\n"}, + {}) + before = {p: p.read_bytes() for p in root.rglob("*.md")} + _intake.reconcile(root, source_reader=_reader( + {"_a_helper": ["src/a.py:1"], "_b_helper": ["src/b.py:2"]})) + assert {p: p.read_bytes() for p in root.rglob("*.md")} == before + + +def test_a_multi_repo_target_is_refused_not_guessed(tmp_path): + """`workspaces`, `health_fixes`, `priors` and `graphical_ep` are topic + clusters, not repos — and among the largest buckets in draft/. Guessing one + repo for them would produce confident nonsense over the biggest part of the + backlog.""" + root = _mind(tmp_path, + {"bug/health_fixes/a_broken_thing.md": + "# Broken\n\nSee @PyAutoFit/autofit/x.py and @PyAutoArray/y.py\n"}, + {}) + slug, err = _intake.resolve_repo(root, "health_fixes") + assert slug == "" + assert "not a single repository" in err + assert "autofit" in err and "autoarray" in err # names the real candidates + + +def test_a_real_repo_target_resolves_to_its_slug(tmp_path): + """The other side of the same door: a genuine target resolves via the body + map, through the same `normalise_repo` aliases the sizing faculty uses.""" + for target in ("autofit", "PyAutoFit", "pyautofit"): + slug, err = _intake.resolve_repo(tmp_path, target) + assert err == "" + assert slug == "PyAutoLabs/PyAutoFit" From c7c8b17fab4f5d14b22edfc241331978b3a6a773 Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 10 Aug 2026 20:18:34 +0000 Subject: [PATCH 2/2] fix: keep the default reconcile output byte-identical MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Found auditing this branch against main. Widening the band column to fit `needs-review` changed the printed output of the DEFAULT path too — every existing Mind-local run gained six columns of indent for a band that run can never emit. The suspects themselves were unaffected (verified identical against main, 29 of 135, no keys removed), but the text a human reads had shifted for work that has nothing to do with this feature. The width now follows the widest band actually present, floored at the previous 6. A default run prints exactly what main printed; only a run that emits `needs-review` widens. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01PY39MRi1oPbxhhr6tDToQU --- agents/conductors/intake/_intake.py | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/agents/conductors/intake/_intake.py b/agents/conductors/intake/_intake.py index bb602b8..90909dd 100755 --- a/agents/conductors/intake/_intake.py +++ b/agents/conductors/intake/_intake.py @@ -1080,13 +1080,18 @@ def emit_reconcile(res: dict): if not res["suspects"]: print(" backlog reconciles clean against the complete/ records " "and active/.") + # Column width follows the widest band actually present, so the default + # (Mind-local) run stays byte-identical to what it printed before the + # upstream leg existed — only a run that emits `needs-review` widens. + width = max((len(s["confidence"]) for s in res["suspects"]), default=6) + width = max(width, 6) for s in res["suspects"]: - print(f"[{s['confidence']:>12}] {s['path']}") + print(f"[{s['confidence']:>{width}}] {s['path']}") for f in s["findings"]: ev = f["evidence"] if len(ev) > 160: ev = ev[:157] + "…" - print(f" {f['kind']}: {ev}") + print(f"{' ' * (width + 3)}{f['kind']}: {ev}") print("\nRetiring a prompt stays human: verify against the target repo's " "git log / merged\nPRs, then retire it to the complete/ archive by hand.") if up: