Skip to content

feat: intake reconcile --repo — read the target repo for staleness evidence - #224

Merged
Jammy2211 merged 2 commits into
mainfrom
feature/reconcile-upstream-repo-mode
Aug 10, 2026
Merged

feat: intake reconcile --repo — read the target repo for staleness evidence#224
Jammy2211 merged 2 commits into
mainfrom
feature/reconcile-upstream-repo-mode

Conversation

@Jammy2211

@Jammy2211 Jammy2211 commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

Summary

Leg 3 of the draft-staleness work. Closes #223.

Legs 1 and 2 shipped Mind-local signals — the PyAutoMind lifecycle.py gate keys, and the intake reconcile re-rank that cut the flag rate from 96/148 to 29/135. Both are structurally blind to two of the five findings the 2026-08-09 draft/ sweep confirmed: one whose evidence sat in a sibling prompt rather than a completion record, and one whose fix shipped upstream with no completion record written at all. The parent prompt measured that no amount of re-ranking reaches those — every attempt cost precision without gaining truth. Only reading the target repo can. This adds that read.

intake reconcile --repo TARGET resolves the target through PyAutoMind repos.yaml plus the sizing faculty's existing repo_aliases, makes a cached shallow clone, greps for the identifiers each prompt names, and reports them as file:line evidence in a needs-review band. The resolved sha is printed so a verdict is re-checkable.

The trap it is built around

Upstream hits deliberately do not feed overlap_score. draft/bug/autofit/test_mode_bypass_ordered_assertion_ties.md names five identifiers — FitException, check_assertions, ignore_assertions, instance_for_arguments, instance_from_vector — and all five are on PyAutoFit main. The prompt is confirmed NOT shipped: main catches exc.FitException in the TEST_MODE bypass, which looks exactly like the requested fix, but the catch wraps only the likelihood call while model.instance_from_vector (where check_assertions raises on an ordering tie) sits on the line before the try.

Scoring presence as evidence would rank that prompt high and make the one mis-grade this tool must never make. Verified end-to-end against PyAutoFit fbe9f45d:

        medium  overlap=8.0  upstream=7.5  ep_hierarchical_scale_collapse_moment_match.md
  needs-review  overlap=0.0  upstream=6.0  test_mode_bypass_ordered_assertion_ties.md
  needs-review  overlap=0.0  upstream=4.5  plot_functions_discard_kwargs.md

Surfaced, with its evidence, at overlap_score 0.0 — never called shipped. That is structural, not a tuned threshold: no number of upstream hits can reach a Mind-local band.

API Changes

Additive only; no existing signature or output changes for callers that don't pass the new flag.

  • intake reconcile gains --repo TARGET (opt-in; default path byte-for-byte unchanged).
  • reconcile() gains source_reader= and upstream_meta= keyword arguments, both defaulting to the previous behaviour.
  • New public helpers: resolve_repo(), upstream_reader().
  • New confidence band needs-review, emitted only under --repo.
  • Result dict gains upstream (slug/sha/date) and per-suspect upstream_score.

Test Plan

  • pytest tests/test_intake_reconcile_ranking.py — 16 passed (8 pre-existing + 8 new)
  • pytest tests/ --ignore=tests/test_skill_install.py330 passed
  • Default path unchanged: still 29 suspects of 135 scanned, 7 high / 22 medium
  • Live run against PyAutoFit fbe9f45d — trap lands in needs-review, not shipped
  • --repo workspaces|priors|health_fixes|graphical_ep → exit 5, naming real candidates
  • Alias resolution: autofit / PyAutoFit / pyautofitPyAutoLabs/PyAutoFit
  • bin/check_skill_line_counts.sh — within the 200-line mandatory-load budget

Ship gate: RED, on a pre-existing failure unrelated to this diff

PyAutoHeart is not reachable from this environment, so the documented fallback gate (pytest -x, any failure = RED) applied and came back RED. Recording it explicitly rather than quietly passing:

  • Reason: assert "SKIP intake (Codex skill" in result.stdouttests/test_skill_install.py, 2 failures.
  • Not caused by this diff. Both reproduce at merge-base a50efc3 with the diff entirely absent. The assertion is about the installer symlinking skill directories into a Codex home, not about the skills/intake/intake.md edit here. bin/install.sh takes its web-github / ci-only branch (:282) because $HOME/Code/PyAutoLabs does not exist, so it never performs the work the test asserts on.
  • A human acknowledged the RED reason and authorized opening this PR on that basis.

Design notes — two decisions worth reviewing

Noise filter: two named classes, not a spread threshold

The first real run matched TypeError (37 files in PyAutoFit) and autofit_workspace (26) — builtins and repo names, pure noise. Filtering by upstream file-spread was tried and rejected: the counts do not separate, because instance_from_vector is a real signal at 22 files, just under autofit_workspace at 26. Any threshold dropping the noise also drops one of the trap's own identifiers. Filtered the two classes explicitly instead; one all-noise prompt dropped out of the results.

Plain shallow clone, not treeless

The parent prompt suggested --depth 1 --filter=blob:none. That is wrong here — this leg greps source, and a treeless clone refetches every blob on demand to answer that. Plain --depth 1 with GIT_LFS_SKIP_SMUDGE=1, cached under $PYAUTO_BRAIN_CACHE (default ~/.pyauto-brain/upstream). Full run against PyAutoFit takes ~3.6s warm.

First network access in PyAutoBrain

Nothing under agents/ previously used urllib, requests, gh or git clone. This is strictly opt-in: without --repo no clone, socket or subprocess runs, and test_default_path_makes_no_network_access monkeypatches both socket.socket and subprocess.run to detonate if that ever changes. AUTONOMY.md records the surface and states that widening it is a new decision.

Multi-repo targets exit 5 rather than guessing — workspaces (23 prompts across four work-types), health_fixes, priors and graphical_ep are among the largest buckets in draft/, so a guess would be confidently wrong at scale.

Tests — 8 new, all hermetic

Nothing under tests/ clones — the upstream read is injected through the source_reader seam, preserving the file's existing property that every fixture is a fictional Mind tree in tmp_path.

  • test_upstream_presence_never_produces_a_shipped_verdict — the trap, in miniature
  • test_upstream_evidence_cannot_inflate_a_mind_local_band — the structural defence behind it
  • test_one_upstream_identifier_is_not_a_signal
  • test_builtins_and_repo_names_are_not_upstream_evidence
  • test_default_path_makes_no_network_access
  • test_upstream_mode_still_never_writes
  • test_a_multi_repo_target_is_refused_not_guessed
  • test_a_real_repo_target_resolves_to_its_slug

Generated by the PyAutoLabs agent workflow.

…idence

Leg 3 of the draft-staleness work. Legs 1 and 2 shipped Mind-local signals:
the lifecycle.py gate keys, and the reconcile re-rank that cut the flag rate
from 96/148 to 29/135. Both are structurally blind to two of the five
findings the 2026-08-09 sweep confirmed — one whose evidence sat in a sibling
PROMPT rather than a completion record, and one whose fix shipped upstream
with no record written at all. Re-ranking cannot reach those; only reading
the target repo can. This adds that read.

`intake reconcile --repo <target>` resolves the target through the body map
plus the sizing faculty's existing aliases, makes a cached shallow clone,
greps for the identifiers each prompt names, and reports them as file:line
evidence in a `needs-review` band. The resolved sha is printed so a verdict
is re-checkable.

Upstream hits deliberately do NOT feed `overlap_score`. The reason is a
measured trap: test_mode_bypass_ordered_assertion_ties names five
identifiers, all five ARE on PyAutoFit main, and the prompt is NOT shipped —
main's catch wraps only the likelihood call while the line that actually
raises sits before the `try`. Scoring presence as evidence would rank that
prompt `high` and make the one mis-grade this tool must never make. Verified
end-to-end against PyAutoFit fbe9f45d: it lands in `needs-review` with
overlap_score 0.0, surfaced but never called shipped.

Two noise classes were found on the first real run and filtered: Python
builtins (`TypeError`, 37 files) and repo names (`autofit_workspace`, 26).
Filtering by upstream file-spread instead was tried and rejected — the counts
do not separate, since `instance_from_vector` is a real signal at 22 files.

This is the first network access in PyAutoBrain, so it is strictly opt-in:
without --repo no clone, socket or subprocess runs, and a test detonates on
any attempt. AUTONOMY.md records the surface and that widening it is a new
decision. Multi-repo targets (`workspaces`, `priors`, …) exit 5 naming the
real candidates rather than guessing one — they are the largest buckets in
draft/, so a guess would be confidently wrong at scale.

Tests: 8 new, all hermetic via an injected source_reader — nothing under
tests/ clones. 330 passed. Plain --depth 1 over the treeless clone the
parent prompt suggested: grepping source refetches every blob on demand.

Closes #223

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PY39MRi1oPbxhhr6tDToQU
Found auditing this branch against main. Widening the band column to fit
`needs-review` changed the printed output of the DEFAULT path too — every
existing Mind-local run gained six columns of indent for a band that run can
never emit. The suspects themselves were unaffected (verified identical against
main, 29 of 135, no keys removed), but the text a human reads had shifted for
work that has nothing to do with this feature.

The width now follows the widest band actually present, floored at the previous
6. A default run prints exactly what main printed; only a run that emits
`needs-review` widens.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PY39MRi1oPbxhhr6tDToQU
@Jammy2211
Jammy2211 merged commit f518b24 into main Aug 10, 2026
2 checks passed
Jammy2211 added a commit that referenced this pull request Aug 10, 2026
…r tests (#226)

Two of the three defects in #225, all one root cause: helpers resolving paths
under $HOME/Code/PyAutoLabs, which no cloud/web/CI session has.

worktree_check_conflict FAILED OPEN. worktree_list_claimed returned 0 with
empty output when active.md could not be resolved under $PYAUTO_MAIN, so the
guard could not tell "nothing claims this repo" from "I could not read the
registry" — and answered the former. start_dev step 6 and start_library step 1
both document this call and act on its answer, so a session whose roots are not
at the default path got a green light it never earned. Reproduced during #224:
a conflict check was recorded as clean having read nothing.

Split into worktree_registry_path so the listing can signal failure in its exit
code; the guard now reports CANNOT VERIFY with the paths it tried and returns 3.
--allow-missing-registry proceeds unguarded and says so, never by default.

test_missing_active_md_yields_no_claims asserted exactly the fail-open
behaviour, so it is rewritten to the corrected contract with the reason
recorded — it was pinning the defect. The sibling worktree_claim_is_stale is
documented fail-open by design and is untouched: it resolves active.md itself.
Both fail toward safety.

test_skill_install.py: two tests depended on the checkout being NAMED
PyAutoBrain one level under PYAUTO_ROOT (default bin/../..). They now pin a
fixture root, testing the installer rather than the checkout layout. This is
what made the ship_library fallback gate spuriously RED on #224.

pytest tests/ is now 331 passed with nothing ignored — green in a cloud session.

Closes #225
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: intake reconcile --repo TARGET — read upstream for staleness

2 participants