Complete the reqdrive roadmap: L0 → L3, all Tier 2 items - #1
Merged
Conversation
Covers the L2->L3 readiness gap and all remaining Tier 2 items, with Tier 3 deferred and reasons recorded. Three key findings from measuring the current state: - tests/simple-test.sh sets `set -e` and every assertion is a bare subshell, so test_result's FAIL branch is unreachable. "0 failed" is guaranteed by construction and red-first TDD is impossible today. Fixing this is P0, and the obvious fix is wrong: bash suppresses errexit inside a subshell used as an `if` condition, even when the body sets `set -e` itself. - The draft-PR gate fail-opens three ways, not one: null verification, missing prd.json, and stories with `passes` omitted. The gate is inverted to fail-closed. - The prompt builder's unquoted heredoc leaves stray backslashes in the commit message the agent is instructed to use. Passed three rounds of adversarial critique (38 findings); the log is in the document.
35 bite-sized tasks across 9 phases, each ending in an independently testable deliverable with a commit. Ordering is forced by the foundation: the test harness cannot report a failure today, so P0 repairs it (and proves the repair by mutation, because assertion inversion cannot tell the correct fix from the broken one). Only then does the spec retrofit, the freeze, and any behavior change land. Phases: P0 harness truthfulness, P1 spec retrofit, P2 freeze gate, P3 pipeline harness, P4 fail-closed draft gate, P5 documentation gates, P6 heredoc fix + verify command, P7 policy cluster, P8 close out. Estimated 9-12 working days.
shellcheck is not installed natively here; the wrapper runs koalaman/shellcheck:stable so the plan's lint commands work locally and verify before CI rather than after. Also records the two pre-flight conflict resolutions: red-first tasks squash red into green so every commit is green, and Task 34's word-splitting is to be resolved with a read loop rather than a blanket SC2086 disable.
cd "" returns 0 and stays in the invocation directory, so an empty TEST_TEMP would make the check_git_repo assertion run rm -rf .git in the repo root. set -e was masking a failed mktemp; the next commit removes that protection, so make the guard explicit first.
set -e at the top of the suite aborted the script on a failing subshell before test_result could run, so the FAIL branch was unreachable and '0 failed' was guaranteed by construction. The obvious fix does not work: bash suppresses errexit inside a subshell used as an if condition, and the suppression propagates into the body even with an explicit set -e. The only form that preserves the semantics is set +e at top plus set -e as the first statement of each body, invoked as a simple command.
Assertion inversion cannot validate the errexit fix — inverting a body's last line flips the status under both the correct and the broken form. Mutation discriminates: impl-prompt-return1 yields 3 FAILs under the correct harness and 0 under the broken one.
Two of the three implementation-prompt assertions were pure negatives that an empty file satisfies, so a silent mutant (empty output, success status) was caught by only 1 of 3. Positive content checks raise that to 2 of 3. Remaining weak assertions are recorded in tests/FINDINGS.md rather than frozen silently.
Task 4 (2521737) appended positive grep checks after the two `! grep` negations in the implementation-prompt sanitization assertions. That made the negations non-terminal in their subshells; under `set -e`, bash exempts `!`-prefixed commands from errexit, so a violated negative (pattern present when it must be absent) would report PASS instead of failing the assertion. shellcheck SC2251 flagged both. Convert both to `if grep ...; then exit 1; fi` guards, which participate in errexit regardless of position and can't be re-masked by future appended lines. Update FINDINGS.md F1 to reflect the fix.
The names claimed coverage of 'all codes'. Codes 9 and 10 arrive in P6/P7, and after the freeze lands a rename is a NEEDS_HUMAN event — so rename now, while the rename surface is declared zero. Bodies are unchanged; they enumerate 0-8 and remain correct as a subset check.
Every story now names the runtime test that proves it. Names come from an actual run because source and runtime text differ for 4 of 158 assertions. Stories covering several tests were split so the mapping is one story, one test, one criterion.
30 assertions covering run state, checkpoints, story selection, prompt builders, the completion hook, iteration-summary extraction and implementation-prompt sanitization had tests but no written criterion. Also fills 8 straggler assertions into the existing config.sh (US-CFG-14..19) and schema.sh (US-SCH-33..34) sections. spec-map.sh mapped count: 86 -> 124, zero PHANTOM, zero AMBIGUOUS.
Marks the two claude-gated stories explicitly - they run as test_result where claude is installed and as test_skip under the same name where it is not.
Modules 7-11 (preflight, pr-create, init, review phase, validate + harness) close the remaining gap. spec-map.sh now exits 0: every runtime test name maps to exactly one story, with no phantom or ambiguous entries. Also fixes a stray leading digit and stale module count in the file header.
Plan Task 9 Step 5 — the P1 exit criterion to reproduce F4's count. Measured 18 pure-negative assertions (plan estimated ~21; Task 4's conversion of two negations to if/exit form accounts for part of the gap).
Runtime names come from a real suite run; the parser strips ANSI and splits on the first ': ' only, because all 158 names contain ': ' themselves. Integrity comes from whole-file hashes of the suite and the gate.
Strict precedence: after P0 any FAIL makes the suite exit non-zero, so a truncation rule keyed on the exit code would re-label every weakening as truncation. R0 therefore fires only when the result count is short AND no FAIL was parsed. conditional is a closed enum of one member (claude); an unrecognized value hard-fails rather than exempting, so it cannot be used as a one-word kill switch. Also strips \r from the locked-name extraction: this machine's jq is a native Windows build that writes multi-line -r output in CRT text mode, appending \r before every \n. ran.txt (built by pure bash string parsing) never carries \r, so without stripping, comm would see zero overlap between the two files and misfire R1+R6 on every locked test. Re-accepted the lock so gateSha256 matches this file.
The R7 reporter case is the one that matters: all lib/*.sh emptied and test_result patched to print PASS unconditionally leaves every assertion body byte-identical, which is why a per-body hash was not enough and the freeze is a whole-file hash.
The gate needs no fetch-depth and no base-ref checkout — it compares file hashes against the lock in the same tree, so it behaves identically on a laptop with no remote and in CI.
…F asymmetry Two review findings on the freeze gate, neither a correctness bug: - gate-selftest.sh mut_r2_fail re-ran `oracle-gate.sh --accept` with a comment claiming it stopped R7 masking R2. False: the mutation targets lib/, which is outside the hash surface (only simple-test.sh and oracle-gate.sh are hashed), so R7 never fires here anyway. Removed the dead step (also cuts a ~90s suite run from the self-test) and corrected the comment. mut_r6/mut_r0 keep their re-locks — they DO edit the hashed suite file, so theirs are necessary. - oracle-gate.sh: documented why the multi-line jq pipeline needs `tr -d '\r'` (native Windows jq emits CRLF; a pipe does no newline stripping) while single-value `$(jq ...)` substitutions do not (command substitution strips the trailing CRLF wholesale). The asymmetry read as a latent bug; it isn't. Comment-only; lock re-accepted for the new hash. R2 path re-verified: breaking a lib fn yields GATE FAIL [R2], not [R7].
Nothing invoked run_pipeline before this. Three fidelity traps are handled explicitly: ph_run sets pipefail (lib/run.sh sets bare set -e, but agent failure is detected through a claude|tee pipeline), it captures run_pipeline's exit rather than its return, and README now documents the undocumented timeout dependency. ph_setup also wires a real local bare "origin" remote, since create_pr's unconditional `git push -u origin` would otherwise fail before gh pr create is ever invoked, short-circuiting the very path this test exists to exercise.
The bare-repo origin was $PH_ROOT/../ph-origin.git — a shared sibling
across every ph_setup call under the same TEST_TEMP. Because ph_setup
hard-codes REQ-01 (branch reqdrive/req-01), a second invocation in one
suite run pushed the same branch to the same shared repo and was
rejected, so the pipeline never reached gh pr create and the fake-gh
log came back empty. Task 17 adds four ph_setup/ph_run pairs to one
file and would have hit this immediately, making every --draft grep a
false negative unrelated to the draft logic under test.
Namespaced to ${PH_ROOT}-origin.git so each case gets its own remote.
Verified: two ph_setup/ph_run pairs in one process both reach pr create.
Gutting build_implementation_prompt used to produce 'ok ... # skip' and a green bats run, so the three e2e tests named as the safety net for the P6 heredoc rewrite could not fail. The deterministic fake agent removes the reason the hatches existed.
'bats green' meant nothing while six tests could skip themselves.
The gate cleared --draft on three separate no-evidence paths: null verification (no testCommand configured, only the literal string "false" was checked), a missing prd.json left holding the "?" sentinel, and stories omitting the optional 'passes' field, which select(.passes == false) never matched (null != false). Enumerating those negatives was a losing game, so the PR is now a draft unless the PRD exists, zero stories remain, and verification positively passed (verification_passed == "true"). final_remaining is now an integer (0 by default) and prd_present (0|1) replaces the "?" sentinel as the source of truth for whether a PRD was ever produced. Story counting switched from select(.passes == false) to select(.passes != true) so a story missing the field counts as incomplete rather than complete. verification-summary.json keeps emitting remaining: null when no PRD exists, and gains prd_present so the two cases stay distinguishable. Also removed the Phase 1 hard-abort when the agent never produces a PRD after its planning retries. That path previously called exit EXIT_AGENT_ERROR before Phase 2/3 ever ran, so a run with no PRD produced no PR at all rather than the draft PR the new gate is meant to guarantee for human review. Phase 2 and Phase 3 already tolerate a missing prd.json gracefully (select_next_story returns empty, prd_present stays 0), so this now falls through to a draft PR instead of a silent, evidence-free failure. Adds four red-first tests under "Draft Gate" proving all three fail-opens plus a positive control that a run with full evidence (PRD complete, testCommand passing) still produces a non-draft PR — confirming the fix doesn't just force --draft unconditionally. New BEHAVIOR-SPEC stories US-DRAFT-01..04 keep spec-map total at 163/163; oracle.lock.json re-accepted at 163 tests.
2362488 inverted the draft-PR gate to fail-closed (correct, kept) but also removed Phase 1's abort when planning never produces a valid prd.json, letting the pipeline fall through to an empty draft PR instead. Restore the original abort: write_run_status "failed", run_completion_hook, exit EXIT_AGENT_ERROR. The prd_present=0 gate branch is now reachable only if an agent deletes prd.json mid-implementation (after planning succeeded) — recorded as F6 in tests/FINDINGS.md. Retarget "draft gate: missing prd.json forces draft" (US-DRAFT-02) to assert the restored abort directly: noprd mode now exits 5 with no "pr create" in the gh log, renamed to "draft gate: planning failure aborts with no PR". Re-locked via oracle-gate.sh --accept.
select_next_story used select(.passes == false and ...) while Phase 3's story counting used select(.passes != true). A story omitting the optional passes field entirely was never selected for implementation (== false doesn't match a missing/null field) yet Phase 3 counted it as incomplete, so the PR would draft forever with no way to make progress on that story. Change the predicate to .passes != true to agree with Phase 3. Adds a red-first regression test (US-RUN-31) and closes F7 in tests/FINDINGS.md.
testCommand defaults to empty, so after the fail-closed inversion a default-configured project gets a draft on every run. Preflight now says so at run start and the PR body distinguishes 'no test command configured' from 'tests failed' — which is what makes the tri-state worth carrying rather than collapsing to a boolean.
The dispatch block accepts nine commands; README documented seven. The test parses the live case block, so adding a command in a later phase reddens the suite until README catches up.
Second doc-coverage rule: every REQDRIVE_* config field in lib/config.sh must appear in README, minus a DOC_EXEMPT list of the three derived runtime paths (REQDRIVE_MANIFEST, REQDRIVE_PROJECT_ROOT, REQDRIVE_ROOT). Failed on maxStoryRetries and reviewCommand until documented. US-DOC-02. (Subagent completed the edits; controller ran the final --accept and gate to avoid the park-on-background-job issue with the slow suite.)
Third doc-coverage rule: every accepted CLI flag (parsed from bin/reqdrive's option case-labels, so the --help inside the usage-error string is not a false positive) must appear in README's Run Options. US-DOC-03. The flag was already mentioned parenthetically, so this rule's own red-first was trivial; its value is as a standing gate — when a later task adds --ref, the suite reddens until README documents it. Lock regenerated deliberately by the controller (--accept) and gate-verified 170/170 after a background-job race left the on-disk lock ambiguous.
The audit says reqdrive 'never verifies outputs'. lib/run.sh:1106-1117 re-runs testCommand and reads the real exit code, so that is false. The reasoning that produced the roadmap is kept; only the claim is corrected, in place and dated.
Cases 2/5/7/8 (status of a finished run, exit-code reporting, completion hook env, re-launch) assert on run.json state transitions and join the main suite as US-LAUNCH-01..04. Cases 1/4/6 (detached launch, duplicate block, crash detection) need real background processes and PID liveness — unreliable under MSYS2 per CLAUDE.md — so they run in a new Linux-only tests/launch-lifecycle.sh CI job rather than a lock exemption. Case 3 (logs) asserts process behavior. LAUNCH-TEST-PLAN.md becomes a pointer to the automated coverage.
P5 review findings: - The flag doc-coverage test extracted flags from hardcoded line ranges (sed -n '90,130p;395,425p'), so when bin/reqdrive's option blocks shift — exactly what Task 30 does by adding cmd_verify — a new --ref case label falls outside the window and is silently never checked, letting --ref ship undocumented while the test reads green. Replaced with a whole-file scan for flag case-labels (the )$ anchor still excludes the --help inside the usage string). Proven: an undocumented --ref now reddens the test. - tests/launch-lifecycle.sh killed only the top-level nohup PID, leaking the timeout/claude/tee children (reparented to PID 1). Added kill_tree() (recursive pgrep -P walk); pgid-kill was unsafe because cmd_launch uses plain nohup and shares the test's process group.
Characterization, not red-green: locks build_implementation_prompt's
current output so Task 26's quoted-heredoc rewrite can be proven
unchanged and Task 28 can change it deliberately with an enumerated diff.
The fixture carries every hazard the rewrite could break: &, backslash,
backtick, $, and a literal @@STORY_ID@@ forgery attempt. The golden
captures the current stray-$ escaping defect as-is (Task 28 fixes it).
The golden is canonical LF and the assertion CR-normalizes both sides:
native Windows jq emits the criteria join("\n") as \r\n in text mode, so
the output carries a CR on Windows and none on Linux — a line-ending
artifact, not a semantic difference. US-RUN-32.
Byte-identical output (modulo CR), proven by the golden file. Swaps the
unquoted heredoc — which expanded ${vars} and was a shell-injection
surface — for a quoted heredoc plus explicit ${tpl//@@token@@/"$val"}
injection. Three mechanism traps handled: all 24 backslash-escaped
backticks de-escaped (a quoted heredoc does no escape processing);
replacements quoted so bash >= 5.2 does not expand & in a value to the
matched text; shopt -u patsub_replacement suffixed with || true because
the option does not exist before 5.2 and set -e would abort on its exit 1.
Substitution order puts @@STORY_ID@@ first so a value containing that
literal token (the fixture's forgery attempt) is injected afterward and
never re-matched — reproducing the old single-pass heredoc behavior. The
robust @@-stripping forgery guard is added in the next commit (Task 27),
which deliberately changes the golden; this commit is the pure
mechanism swap. The stray-$ escaping defect is preserved here and fixed
in Task 28.
The quoted-heredoc rewrite injects via ${tpl//@@token@@/"$val"}, so a
PRD value containing a literal @@token@@ could — in a mutual-reference
case that substitution ordering alone cannot fully cover — forge a
placeholder a later pass expands. Stripping @@ from every injected value
before substitution makes that impossible, order-independently.
This is the one deliberate change to the golden the Task 26 refactor
preserved: the fixture's forgery attempt in the description,
'@@STORY_ID@@', is now defanged to 'STORY_ID'. Enumerated golden diff:
- **Description:** Covers @@STORY_ID@@ forgery ...
+ **Description:** Covers STORY_ID forgery ...
(exactly one line; no other output changed.)
Three assertions pin the guards: the shopt || true survives a bash
without patsub_replacement (US-RUN-33), PRD content cannot forge a token
(US-RUN-34), and an ampersand in a title is not expanded to the match
(US-RUN-35).
sanitize_for_prompt escapes $ to \$ for the OLD unquoted heredoc. The
heredoc is quoted now, so the backslash was pure noise reaching the agent
— including inside the commit message it is instructed to use. Un-escape
at injection time (the correct bash form is ${var//\$/\$}; the
naive ${var//\$/$} does not un-escape). lib/sanitize.sh is unchanged:
its backtick neutralization is load-bearing and it has other callers.
Enumerated golden change (4 lines, all \$ -> $):
- **Title:** ...\/c/Users/barclay... -> ...$HOME...
- **Description:** ...\... -> ...${VAR}...
- criterion: Check \/c/Users/barclay -> Check ${HOME}
- commit message: feat: [...] ...\/c/Users/barclay -> ...$HOME
Updated Task 4's escaped-form assertion to the un-escaped form, added
US-RUN-36 (a $ title reaches the agent verbatim, commit line clean), and
corrected US-RUN-30's now-stale prose.
run_pipeline's inline Phase 3 becomes three shared functions so the new reqdrive verify command (next commit) reuses one implementation: verify_collect -> VERIFY_STORIES_* + VERIFY_PRD_PRESENT globals verify_run_tests -> tri-state 0 pass / 1 fail / 2 not-configured verify_write_summary <agent_dir> <req_id> <max_iterations> <mode> Named verification.sh, not verify.sh, to stay distinct from the archived archive/v1-complex/lib/verify.sh. max_iterations is an explicit parameter (a run_pipeline local interpolated into the JSON; omitting it emits a malformed "max": ). The summary is written temp-file + mv (atomic). verify_run_tests's return is captured with || verify_rc=$? because a bare non-zero return would trip set -e before the case ran. Characterization: verification-summary.json keeps its full shape (US-PIPE-02), draft gate behavior unchanged.
Re-runs verification for an existing run and updates its verification-summary.json in merge mode, so re-verifying preserves the iterations/tests/commits evidence trail the PR body renders. Refuses when the run's PID is still alive (EXIT_CONCURRENT_RUN=10), when the checkout does not match the run's recorded branch and no --ref is given (EXIT_GIT_ERROR=4), and when the run or its summary is missing (EXIT_CONFIG_ERROR=3). Exits 0 on pass, EXIT_VERIFICATION_FAILED=9 on fail. Documented verify + --ref in README — the P5 doc-coverage gates reddened on both until documented, which is the gate working. Found a pre-existing latent bug (logged as F8): write_run_status writes pr_url into run.json without JSON-escaping, so an embedded newline makes the file invalid JSON and crashes jq consumers under set -e. verify's pid-read is fail-open to survive it; the root fix in write_run_status is deferred (it touches frozen run_status tests).
Three review findings, each with a red-first test: - cmd_verify silently no-op'd its branch guard when checkpoint.json was absent (a run with empty userStories / maxIterations=0 writes no checkpoint), so verify would record an unrelated branch's evidence. Now refuses with EXIT_CONFIG_ERROR. - --ref checkout failure aborted with git's raw exit 1, contradicting the documented 'exit 4 on branch mismatch'. Now guarded to EXIT_GIT_ERROR. - F8 root cause: write_run_status wrote pr_url into run.json without JSON-escaping, so an embedded newline made the file invalid JSON, crashing every jq consumer (verify's pid guard, the status command). Now escaped via jq -Rn. verify keeps a defensive fail-open too. Fixing F8 (making run.json valid) surfaced a latent test flaw the freeze gate caught (R2): two verify tests passed at 185 only because the invalid JSON made verify's fail-open bypass the concurrency guard. With valid JSON the guard correctly fires on run.json's pid — which in the harness is $$ (the live test runner). Both tests now set a dead pid to reflect a completed run's dead process; 'merge preserves evidence' was passing vacuously (verify refused) and is now a real merge test.
Aligns reqdrive validate's failure exits to EXIT_CONFIG_ERROR (3) so the policy validation Task 32 adds isn't the only field whose malformation exits 3 while every other field exits 1. Closes F5 (the existing assertion checked only -ne 0, never pinning the code) with two exit-code assertions (US-VAL-03/04). Also fixed a latent set -euo pipefail bug: the schema-error-display 'validate_config_schema ... | while read' pipeline failed under pipefail and aborted cmd_validate before the exit code was set, so a type violation exited 1 regardless. Terminated the loop with || true.
policy lives inside reqdrive.json (one file, one loader, one validator),
not a separate policy.json. Schema validates it: policy must be an object,
policy.scopeCheck (if present) must be warn|block, policy.riskTiers values
must all be arrays. reqdrive_load_config exports REQDRIVE_POLICY_JSON and
REQDRIVE_POLICY_SCOPE_CHECK (defaults {} / warn). Tasks 33-34 consume these.
The config doc-coverage test would derive false field names from the two
new REQDRIVE_POLICY_* vars, so they're added to its DOC_EXEMPT list as
derived from the single documented policy field. US-POL-01..04.
reqdrive_load_config still does NOT schema-validate on load (deferred, per
the design) — reqdrive validate remains the validation entry point.
lib/policy.sh classifies a path into high/medium/low/none. Patterns are bare path prefixes, NOT globs: in bash [[ ]] matching globstar does not apply, so ** and * are indistinguishable and both cross /, and src/auth/** fails to match src/auth itself. A path matches when it equals the pattern or begins with pattern/ — so src/auth.sh does NOT match src/auth (the trap a glob would have hidden), while src/auth/login.ts does. Tiers are probed high->medium->low so the highest wins. The jq-per-tier read strips a trailing CR (native Windows jq emits CRLF in multi-line -r output; a no-op on Linux). US-POL-05..08.
After each implementation iteration, git diff --name-only HEAD~1 HEAD is classified by risk tier; a high-risk path changed in an iteration whose tests did not pass is a finding. warn (default) logs it to scope-findings.txt and the PR body and continues; block aborts the iteration with EXIT_PREFLIGHT_FAILED. The touched paths are iterated with a while-read loop, not $changed word-splitting, so filenames with spaces are safe and there is no SC2086. Ships warn-only: the roadmap wanted a hard gate, the architecture principle says warn before enforce. The knob makes the gate one config edit away, and warn-mode data is what would justify flipping the default. US-SCOPE-01..03.
P7 review: a risk-tier pattern written as a directory with a trailing
slash ('src/auth/' — a natural way to name a directory) matched NOTHING,
silently defeating the scope check for that path. policy_tier_for_path now
strips a trailing slash so 'src/auth/' behaves as 'src/auth'. For a
security-relevant gate a silently-inert high-risk pattern is a real
footgun. US-POL-09.
Also documented the HEAD~1 invariant in policy_scope_check: it always
resolves because preflight's check_base_branch_exists guarantees the base
branch (and thus >= 1 commit) exists before the work branch is cut; if
that is ever weakened the check fails open rather than erroring.
…eCheck
Final whole-branch review findings, both closed:
- The freeze gate hashed only simple-test.sh + oracle-gate.sh, but the
suite sources tests/lib/pipeline-harness.sh at enforce time, so its
content decides ~10 tests' outcomes. Gutting the harness toward
fake-success (ph_run(){ echo 0; } + a canned pr-create log) made those
tests pass for the wrong reason with the hash unchanged and the gate
green — the exact hole R7 closed for test_result. Added harnessSha256
and specmapSha256 to R7. Proven: the fake-success exploit now fires
GATE FAIL [R7]. (F9)
- REQDRIVE_POLICY_SCOPE_CHECK came from a single-value jq without a
CR strip; a surviving CR (native Windows jq) would make [ mode = block ]
false and silently downgrade the hard scope gate to warn. Strip it.
FINDINGS: F9 closed; F3 re-triaged to essentially-zero exposure per the
review (story_json is schema-validated and aborts under set -euo pipefail
rather than emitting blanks).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Takes reqdrive from readiness rung L0 to L3 and completes every remaining CLAUDE.md Tier 2 item. 35 tasks across 9 phases, each landing behind a frozen, tamper-evident test oracle; Tier 3 is explicitly deferred with recorded reasons.
Why this was L0, not the "L2, docs-only gap" the survey assumed
Three findings measured at the start, each verified in-tree:
set -eplus the bare-subshell +test_result "..." $?shape meant a failing assertion killed the script beforetest_resultran — so "0 failed" was true by construction and red-first TDD was impossible. Fixed in P0, mutation-proven.prd.json,passes-omitted stories) — a PR could present as ready-to-merge with no evidence. Inverted to fail-closed in P4.Phases
reqdrive verifyVerification
tests/oracle-gate.sh→ 202/202, suite exit 0. The freeze hashessimple-test.sh,oracle-gate.sh,pipeline-harness.sh, andspec-map.sh; any change to the test infrastructure isNEEDS_HUMAN.bash -nclean acrossbin/,lib/, and the test scripts.The gate caught a real regression mid-effort (a concurrency-guard test that had been passing vacuously), and the final adversarial review closed a genuine freeze-surface hole — both are documented in
tests/FINDINGS.md.Deferred (recorded in CLAUDE.md Decision Log)
Tier 3 (each a separate effort): vision QA (Node/Python + Playwright), worktree
orchestrate, PR-rejection feedback, CI polling, cost tracking, adaptive retries. Also: config-load-time schema validation, and making the review agent a genuine writer≠grader.Design trail
docs/superpowers/specs/2026-07-23-reqdrive-roadmap-completion-design.md(spec, 3 adversarial critique rounds) anddocs/superpowers/plans/2026-07-23-reqdrive-roadmap-completion.md(the 35-task plan). Readiness correction owed to the external WORKFLOW.md survey is recorded indocs/STATUS.md.🤖 Generated with Claude Code