fix: pre_build must not stage untracked human work - #233
Merged
Conversation
run_workspace ran `black "$d/"` then `git add "$d/"` over scripts/, notebooks/ and slam_pipeline/ across 13 repos. Both operations reach untracked files, so any uncommitted work in a workspace's scripts/ was reformatted on disk and pushed inside the "pre build" commit — to a public repo, silently, exit 0. Same leak class as #126, which was fixed for dataset/ and config/ by dropping their staging (#156) while leaving the scripts/ path open. Near-miss during the 2026-08-07 release, caught only because the operator moved the file out by hand. Reproduced against the pre-fix script on throwaway fixture repos: the private file was committed and pushed to the remote, exit 0. Two legs: - A fail-fast preflight sweeps every repo for untracked files under the directories the run mutates, before the first repo is touched. It must precede everything: run_workspace commits and pushes each repo before moving to the next, so a per-repo check aborting midway would leave earlier repos already published. It also aborts on a missing checkout, which previously surfaced as a bare `cd` error partway through. This answers the open atomicity question in docs/pre_build_failure_audit.md §6. - Staging narrows to `git add -u` (tracked edits and deletions) plus files created by the run, added by explicit path — so the directory-wide form cannot return. New notebooks from generate.py are still staged, which is why `git add -u` alone is insufficient. The repo list moves into a WORKSPACE_SPECS array: two passes now read it, and a second hand-maintained list would drift out of step. No --allow-dirty override — an override is precisely the operator vigilance this replaces. Text assertions cannot prove a gate fires, so tests/test_pre_build_staging.py runs the real script against fixture git repos with black/python/gh stubbed and real bare remotes. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X13iRHCuuLptM7XsT1tfdH
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #232.
run_workspaceranblack "$d/"thengit add "$d/"overscripts/,notebooks/andslam_pipeline/across 13 repos. Both operations reach untracked files, so any uncommitted work in a workspace'sscripts/was reformatted on disk and pushed inside the"pre build"commit — to a public repo, silently, exit 0.Same leak class as #126, which was fixed for
dataset/andconfig/by dropping their staging (#156) while leaving thescripts/path open. Near-miss during the 2026-08-07 release, caught only because the operator moved the file out by hand.Reproduced first, against the unmodified script on throwaway fixture repos with real bare remotes: the private file was committed as
"pre build"and pushed to the remote, exit 0. The same fixture with this change aborts at the preflight, leaves the file byte-identical (md5 unchanged — black never reached it), and creates no commit.What changed
A fail-fast preflight sweeps every repo for untracked files under the directories the run mutates, before the first repo is touched. It has to precede everything:
run_workspacecommits and pushes each repo before moving to the next, so a per-repo check aborting midway would leave earlier repos already published. It reports every offending repo and path in one pass, so one run surfaces the whole problem. It also aborts on a missing checkout, which previously surfaced as a barecderror partway through a run.This answers the open atomicity question in
docs/pre_build_failure_audit.md§6 — "worth a fail-fast pre-pass (all repos validated before any push)?" — which that audit costed as a follow-up. Marked resolved there.Staging narrows to
git add -u(tracked edits and deletions) plus files created by the run, added by explicit path, so the directory-wide form cannot return. New notebooks fromgenerate.pyare still staged — which is whygit add -ualone is insufficient.The repo list moves into a
WORKSPACE_SPECSarray. Two passes now read it, and a second hand-maintained list would drift out of step — the preflight would then silently skip a repo it exists to protect.No
--allow-dirtyoverride, deliberately: an override is precisely the operator vigilance this replaces.Files
pre_build.sh— the two legs above.tests/test_pre_build_staging.py(new) — runs the real script against fixture git repos withblack/python/ghstubbed and real bare remotes. Text assertions cannot prove a gate fires; these do. Six cases: WIP abort, multi-repo reporting, gitignored files not blocking a release, generated notebooks still staged, tracked deletions, missing checkout.tests/test_pre_build_skill.py— repo-list regex retargeted at the array; two assertions added (no directory-formgit add; preflight precedes every mutation).skills/pre_build/pre_build.md— the preflight documented as a numbered step.docs/pre_build_failure_audit.md— §6 atomicity question marked resolved.Test state
302 passed / 7 failed / 3 skipped locally. The 7 failures are pre-existing and environmental —
ipynb-py-convertwill not build in this sandbox — and the identical 7 fail on pristinemainwith these changes stashed. All 10pre_buildtests pass.Generated by Claude Code