Fix #97: scope REPORT_DIR to the Test Reports section - #98
Closed
laird wants to merge 1 commit into
Closed
Conversation
regression-test.sh picked the report directory with a global `grep "Location: " CLAUDE.md | head -1`. CLAUDE.md has several such lines and the Unit Tests one comes first, so REPORT_DIR became `tests/test_*.sh` and mkdir -p created a directory with that literal name. That directory shadows the project's own documented test glob: once it exists, `for t in tests/test_*.sh; do bash "$t" || exit 1; done` yields the directory, bash exits 126, and the unit suite goes red with no failing test behind it. Reports also never reached docs/test/regression-reports/. Changes: - regression-test.sh: scope the lookup to the "**Test Reports**" section, and reject a resolved REPORT_DIR still containing glob metacharacters (*?[) — that is a parse failure, not a directory name — falling back to the default. - tests/test_regression_report_dir.sh: new. Asserts no glob-named path is ever created and the report lands in docs/test/regression-reports/. Covers two scenarios: the real CLAUDE.md section ordering, and a Test Reports location that is itself a glob (exercising the metacharacter guard directly). Verification: both assertions confirmed non-vacuous by reverting each guard and observing the matching failure. 18/18 shell unit tests pass; bash -n and py_compile clean. End-to-end run against the real CLAUDE.md now writes to docs/test/regression-reports/ and creates no stray directory. Distinct from #73/#85, which cover unit-test counting and exit codes. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Owner
Author
|
Superseded — master already fixed this (see #97, closed with the master line numbers). Closing without merging to avoid re-applying a stale branch's version of this fix over master's current one. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #97
Root Cause
regression-test.shselected the report directory with a global first match:REPORT_DIR=$(grep "Location: " CLAUDE.md | sed '...' | head -1)CLAUDE.mdhas twoLocation:lines and the Unit Tests one comes first, soREPORT_DIRbecametests/test_*.shandmkdir -pcreated a directory with that literal name.Impact
Worse than a misplaced report — the directory shadows the project's own documented test glob. Once it exists:
yields the directory,
bashexits 126, and the unit suite goes red with no failing test behind it. Reports also never reacheddocs/test/regression-reports/.Solution
**Test Reports**section rather than taking the first global match.REPORT_DIRthat still contains glob metacharacters (*?[) — that is a parse failure, not a directory name — falling back to the default.Changes Made
plugins/autocoder/scripts/regression-test.sh— section-scoped lookup + metacharacter guard.tests/test_regression_report_dir.sh(new) — asserts no glob-named path is ever created and the report lands indocs/test/regression-reports/. Two scenarios: the real CLAUDE.md section ordering, and a Test Reports location that is itself a glob, so the guard cannot rot into dead code.Verification
FAIL: regression-test.sh created a path literally named 'tests/test_*.sh'.bash -nandpy_compileclean.CLAUDE.md: report now written todocs/test/regression-reports/, no stray directory.Scope
Distinct from #73/#85 (unit-test counting and exit codes); this is the
REPORT_DIRselection path only. Independent of #96 — both branch frommasterand touch different files.Note: on this base the harness still reports
0/0 passedas green — that is the #73/#85 defect, unchanged here and still behind the blocked #74 stack.🤖 Generated with Claude Code