fix(regression-test): false-green runs due to three parse bugs - #114
Open
laird wants to merge 24 commits into
Open
fix(regression-test): false-green runs due to three parse bugs#114laird wants to merge 24 commits into
laird wants to merge 24 commits into
Conversation
…g to N-clean; scope + exercise preflight; swarm cadence)
Design spec for adding a human-gated planning capability to the manager session (brainstorm → spec → critical review → decompose to stories → existing fleet implements), plus renaming /fix→/dev and /fix-loop→/dev-loop with aliases. Reuses existing skills and the decomposed/subtask convention; adds two backlog-aware behaviors to the manager loop. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Critical-design-review round 1 approved the planning-pipeline spec as-is: 0 literal-wrongness findings, 0 forced decisions. Empirically confirmed the two load-bearing hand-off facts (subtask is claimable; decomposed parents are excluded from claiming). Recorded the CDR approval in the spec status and carried the three CIR-deferred items forward as notes B8/B9 (+ existing B5). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…eview-1 to autocoder-planning-pipeline-implementation-plan
Tracks the 2026-07-28 handoff documenting the /fix→/dev rename and peters-toolkit:bugfix integration, plus the still-unaddressed critical review of the planning-pipeline implementation plan. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…aim lock until a terminal outcome
…ver reached the ship branch
…ini/droid markers, stale comments)
…rs.sh (GraphQL replay lands new files 644)
… exclusion, drift repair), marketplace to 3.32.0
…ound/improved/learned per round, verdict chapter)
…e v2 removal Atlassian removed POST /rest/api/2/search from Jira Cloud (HTTP 410, CHANGE-2046). list and any-claimable now use POST /rest/api/3/search/jql with an explicit fields list, nextPageToken pagination (no startAt), and existence-of-first-page instead of the removed total for any-claimable. Issue lifecycle endpoints stay on v2 (not removed; plain-string bodies). The fake serves the new contract, enforces a settable page cap to exercise the token loop, and returns 410 on v2 search so regressions cannot pass. Live verification against real Jira Cloud pending by coordinator.
…text Live smoke showed /rest/api/3/search/jql returns description as an ADF document object, not v2's plain string. The list reshape now walks ADF content nodes collecting text (paragraphs joined with newlines; tolerates null, plain strings, malformed nodes), so body stays a plain string for all consumers. v2 CRUD read paths are untouched (still plain strings). The fake now serves ADF descriptions from v3 search to pin the contract.
…mp autocoder 4.10.1 / marketplace 3.32.1
…-smoke-test.sh; live validation pending an ADO sandbox)
…gents The SKILL.md had no model guidance. Top-level coordinator (scope/grade/fix decisions) uses the deep model; subagents and swarm workers use the balanced model. Mirrors the autocoder manager/worker tier pattern. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
… model selection (#110) * feat(autocoder): add model-config startup confirmation and env-var-driven model selection Autocoder agents now resolve model tiers from env vars → .autocoder.json → built-in defaults (claude-opus-5 / claude-sonnet-5 / claude-haiku-4-5 on Claude Code; gemini-2.5-pro / gemini-2.0-flash on Antigravity). On first run with no model config found, the agent presents the defaults and asks the user to confirm or override before proceeding. The confirmed models are exported as MANAGER_MODEL / WORKER_MODEL / FAST_MODEL and referenced in every Task tool call in the workflow so agents actually use them. Also adds skills/autocoder/references/model-config.md documenting the tier system and how to persist overrides via .autocoder.json. Parallel maintenance: plugins/autocoder/commands/fix.md ↔ .agent/workflows/fix.md Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * chore: bump autocoder to 4.11.0, marketplace to 3.33.0 Reflects model-config startup confirmation and env-var-driven model selection added in the previous commit. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
#112) Switch fix.md and model-config.md from full API model IDs (claude-opus-5) to platform tier shorthand (opus/sonnet/haiku for Claude Code, pro/flash for Gemini). Agents inherit credentials from the running session — no separate API keys needed. Rename skills/harden → skills/improve to reflect the general improvement and validation loop, not just security hardening. Bump autocoder 4.11.0 → 4.12.0, marketplace 3.33.0 → 3.34.0. Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
- REPORT_DIR was grabbing the first Location: in the file (the unit-test location tests/test_*.sh) instead of the Test Reports block — causing mkdir -p to create a literal directory named tests/test_*.sh. Fixed by scoping the grep to the **Test Reports** block. - Added a metacharacter guard: if REPORT_DIR contains *, ?, or [, it falls back to docs/test/regression-reports/ and warns. - Added one-time cleanup: removes the literal tests/test_*.sh/ directory left behind by prior buggy runs. - UNIT_TEST_CMD was extracted via "### Unit Tests Only" which doesn't exist in this repo — always empty, so all tests were skipped. Fixed by parsing the "Run all shell tests:" inline-code line in the Test Framework Details block. - Pipeline tee was swallowing the left-side exit code. Restructured to use PIPESTATUS[0] for accurate failure detection. - Added UNIT_FAILED > 0 belt-and-suspenders check so shell-style tests that print "N failed" are always treated as failures. - Fixed count_before to sum all occurrences (awk) instead of tail -1, so multi-file shell test runs are tallied correctly. - Updated CLAUDE.md "Run all shell tests:" to use find -type f so glob expansion never matches the now-removed bogus directory. - Both skipped suites now exit non-zero; zero tests executed is not a pass. - Added tests/test_regression_test_runner.sh covering the all-suites- skipped case and the glob-safe report directory. Closes #73 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
6 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Location:in the file (tests/test_*.sh) instead of the Test Reports block, causingmkdir -pto create a literal directory namedtests/test_*.sh. Fixed by scoping the grep to the**Test Reports**block, adding a glob-metacharacter guard, and a one-time cleanup of the existing bogus directory.### Unit Tests Onlywhich doesn't exist — always empty, so all tests were silently skipped. Fixed by parsing theRun all shell tests:inline-code in the Test Framework Details block, and usingbash -cto execute the extracted shell loop.teewas swallowing the left-side exit code. Restructured to check${PIPESTATUS[0]}. Also added aUNIT_FAILED > 0check for shell-style tests that print results without relying on exit codes.CLAUDE.md"Run all shell tests:" to usefind -type fso the (now removed) bogus directory can never break it again.tests/test_regression_test_runner.shcovering the all-suites-skipped and glob-safe-report-dir cases.Known pre-existing failure (not in scope)
test_issues_gh_search.shhas 11 failures due to aBASH_ENVPATH contamination issue — tracked separately in #113. The runner now correctly surfaces this failure instead of hiding it.Test plan
bash plugins/autocoder/scripts/regression-test.shexits non-zero (was: false green 0)docs/test/regression-reports/(was:tests/test_*.sh/)tests/test_*.sh/bogus directory cleaned upbash tests/test_regression_test_runner.sh→ 2 passed, 0 failedbash -n plugins/autocoder/scripts/*.sh→ syntax OK🤖 Generated with Claude Code