fix(CC-571): sqlite atomic-script gap in _ctx_fts_rebuild + _ctx_index_file - #539
Merged
Conversation
…x_file _ctx_fts_rebuild's DROP+CREATE+INSERT sequence had no BEGIN/COMMIT, and critically no -bail: direct reproduction confirmed the sqlite3 CLI's default behavior on a mid-script SQL error is to print the error and keep executing subsequent statements -- it does NOT stop -- so a bare BEGIN...COMMIT alone would still reach and execute COMMIT after silently skipping the failed statement, committing a half-built content_fts. Both callers (pmctl_context_index, pmctl_context_update) also called it unchecked, reporting success regardless. Fixed by wrapping in BEGIN IMMEDIATE/COMMIT with `sqlite3 -bail`: an error now aborts before COMMIT, leaving the transaction open; the process exit closes the connection, triggering an automatic ROLLBACK that leaves the previous content_fts fully intact. Both callers now check the return value and report an honest "FTS index rebuild failed; stale index retained, LIKE fallback still available" instead of a bare success line -- non-fatal, since FTS is a best-effort acceleration layer. Scope expanded during the mandatory reuse/simplify pass: the altitude review found the identical pattern in _ctx_index_file (pmctl_context_update's other call site), which is more severe since it writes the PRIMARY files/symbols/file_chunks data, not an acceleration layer. Direct testing also found a third, independent bug there: the function's trailing `rm -f "$tmpf"` masked sqlite3's real exit code (the function always returned rm's status, ~always 0, regardless of whether sqlite3 succeeded). Both functions now share one `_ctx_sqlite_exec_atomic` helper (closing a reuse-review finding that the fix reinvented a pattern already established in memory.sh's memory_usage_commit rather than citing/sharing it), and _ctx_index_file's caller now treats its failure as fatal -- unlike FTS, a failed primary-index write must not be reported as "re-indexed". 5 new regression tests cover: FTS rollback preserves the old index, _ctx_index_file's return code reflects sqlite3's real result, and both callers report failure honestly rather than silently claiming success. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0177ds9nxEMtsh3bwpBxNHYx
…ne itself
Gate finding critic-F001 (round 1, gate-20260826-011700-b139d2): printing
the FTS-rebuild-failure diagnostic on stderr while stdout still read a bare
"context index: N indexed, M skipped" / "context update: re-indexed <path>"
is a contradictory summary -- a caller that only looks at stdout, or only
checks exit code 0, saw nothing but success. This violated CC-571's own
Requirement 2 ("failure must not print success-worded text").
Both callers now append a degradation qualifier directly onto the final
summary line when the FTS rebuild fails, in addition to the existing
stderr diagnostic. Updated both regression tests to assert the summary
line itself discloses degradation, not just that stderr mentions it.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0177ds9nxEMtsh3bwpBxNHYx
Gate finding critic-F001 (round 2, gate-20260826-021038-ac0bc2): when content_fts did not exist before a rebuild attempt (a first-time index build), a failed rebuild's rollback leaves no FTS table at all -- the existing "existing (now stale) FTS index retained" message was false in that case, since there was no existing index to retain. Both callers now check whether content_fts existed before attempting the rebuild (pmctl_context_index already captured this via _fts_present; pmctl_context_update now captures it too) and branch the message: "stale index retained" only when one genuinely existed, otherwise "no FTS index available, LIKE fallback only". Strengthened case_context_index_reports_fts_rebuild_failure_honestly (which already exercised a fresh fixture repo -- exactly the first-time-build scenario) to assert the correct branch's wording and that content_fts is actually absent afterward, per the finding's own verification_expectation. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0177ds9nxEMtsh3bwpBxNHYx
screenleon
added a commit
that referenced
this pull request
Aug 26, 2026
…lt path (#541) * docs(BACKLOG): mark CC-571 done, record PR #539 reference Status flip missed at merge time -- PR #539 shipped and merged, but BACKLOG.md's index row and body heading were never flipped from active to done. This is the fourth time this exact pattern has recurred in one session (after CC-567, CC-533, CC-015) -- this time by the same agent that had just fixed the other three, which is its own lesson: the fix has to happen inside the PR before merge, not as a mental note for after. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0177ds9nxEMtsh3bwpBxNHYx * fix(CC-572): pr-gate synthesis retry removes (not truncates) the result path Diagnosed from repeated production protocol failures during the CC-571 gate saga: apply_patch failures ("invalid patch: multiple operations target <file>", "invalid hunk", "Failed to find expected lines") kept recurring on the synthesis correction retry, in both sequential and parallel mode. Root cause: the reviewer-protocol retry (which never showed this failure) always writes to a brand-new path (reviewer-<name>-<ts>-retry1.md) -- always an unambiguous "Add File" for whatever patch tool the executor uses. Both synthesis retries instead re-dispatch to the SAME fixed $OUTPUT_FILE path. The sequential retry already truncated it (`: > "$OUTPUT_FILE"`) before this fix, and STILL hit the identical failure -- confirmed by direct trace inspection that a path which still EXISTS on disk, even at 0 bytes, can lead the executor's patch tool to choose an "Update File" operation (which locates existing content to edit) instead of "Add File". An update against empty/rewritten content then fails outright, since there's no content to locate -- a hard patch-tool failure unrelated to the actual synthesis content. The parallel retry didn't even truncate; it left the first attempt's full content in place. Fixed by removing the path entirely (not truncating it) before both retries, via a new shared `gate_clear_retry_target` helper next to `gate_dispatch_command`. This forces an unambiguous "Add File" the same way the already-reliable reviewer-retry path's brand-new filename does. Added a test-only capture hook (CODEX_GATE_CAPTURE_OUTPUT_EXISTS_DIR) that records, at the very start of each synthesis dispatch, whether the result path exists on disk at that instant -- and two regression tests (one per mode) asserting the first dispatch sees "exists" (reviewer content already written) and the retry sees "absent". All existing synthesis-protocol (14) and sequential-protocol (4) tests still pass. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0177ds9nxEMtsh3bwpBxNHYx * fix(CC-572): add missing test docstrings on the two mode wrappers lint-test-docstrings requires each test_* function to carry its own '# Behavior:' marker directly above its declaration -- the shared helper's docstring didn't satisfy that for the two thin per-mode wrappers, which caused tests/bin/run-all-tests.sh to fail at phase 0 and skip 99 suites. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0177ds9nxEMtsh3bwpBxNHYx * docs(BACKLOG): mark CC-572 done in the same PR that ships it Following the lesson from CC-524/CC-567/CC-533/CC-015/CC-571: flip the status inside the shipping PR, not as a follow-up after merge. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0177ds9nxEMtsh3bwpBxNHYx * docs(BACKLOG): record PR #541 reference for CC-572 Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0177ds9nxEMtsh3bwpBxNHYx * docs(CC-572): trim comments/ticket text to why, not what gate_clear_retry_target's own comment restated what the function does (the name already says that); trimmed to keep only the why (the Update-vs-Add-File patch-tool failure mode). BACKLOG.md's Requirement/ Update sections similarly dropped line-number-level implementation detail that duplicates the code and would go stale as soon as the code moves -- kept only the outcome and the reasoning. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0177ds9nxEMtsh3bwpBxNHYx * docs(CC-572): remove ticket-number references from code comments Ticket refs belong in BACKLOG.md/DECISIONS.md, not in code/test comments, which should describe current behavior on their own terms. Removed all CC-572 mentions from runtime/bin/pr-gate.sh and tests/shell/test-pr-gate.sh (comments and the _cc572_-prefixed variable names), rewording each to state what the code does/why without the ticket citation. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0177ds9nxEMtsh3bwpBxNHYx --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
4 tasks
screenleon
added a commit
that referenced
this pull request
Aug 26, 2026
…ns (#544) * fix(ship): give the permanent-test admission record a PR body template slot Second reading of CC-554's admission-recording rule (effective since pr:#530): across a 12-PR evidence window, both qualifying instances found (pr:#539's 7 new regression cases, pr:#541's 2 new test functions) added a permanent test as a gate-finding remedy but recorded zero admission-criteria citations in the PR body. The rule text lives in Step 2.5, several screens before the PR body actually gets composed in Step 4 -- a free-floating prose reminder read once during remediation is not reliably recalled by the time the PR body template gets filled in. Add an explicit `Permanent test admissions:` field to that template so its omission is conspicuous (fill it in, or write `none`) instead of silent, and add a regression assertion that the template carries the field. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0177ds9nxEMtsh3bwpBxNHYx * fixup: clarify none is not a substitute for an alternative-path record Two gate reviewers (critic, architecture-reviewer) independently flagged that the template's `| none` fallback read as usable whenever no new permanent test was added -- including a round whose finding was resolved via a recorded alternative, which Step 2.5 still requires a line for. Reworded so `none` applies only when no finding this round was assessed against Step 2.5 at all, and added regression assertions for both the alternative-path requirement and the none-is-not-a-substitute wording. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0177ds9nxEMtsh3bwpBxNHYx --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
_ctx_fts_rebuild's DROP+CREATE+INSERT sequence had noBEGIN/COMMIT, and critically no-bail: direct reproduction confirmed the sqlite3 CLI's default behavior on a mid-script SQL error is to print the error and keep executing subsequent statements — it does not stop — so a bareBEGIN...COMMITalone would still reach and executeCOMMITafter silently skipping the failed statement, committing a half-builtcontent_fts. Both callers (pmctl_context_index,pmctl_context_update) also called it unchecked, reporting success regardless.Fixed by wrapping in
BEGIN IMMEDIATE/COMMITwithsqlite3 -bail: an error now aborts beforeCOMMIT, leaving the transaction open; the process exit closes the connection, triggering an automaticROLLBACKthat leaves the previouscontent_ftsfully intact. Both callers now check the return value and report the degradation — including in the final stdout summary line itself, not just stderr (see gate findings below) — instead of an unqualified success message. Non-fatal, since FTS is a best-effort acceleration layer (LIKE fallback remains available).Scope expanded during the mandatory reuse/simplify pass: the altitude review found the identical pattern in
_ctx_index_file(pmctl_context_update's other call site), which is more severe since it writes the primary files/symbols/file_chunks data, not an acceleration layer. Direct testing also found a third, independent bug there: the function's trailingrm -f "$tmpf"masked sqlite3's real exit code (the function always returnedrm's status, ~always 0, regardless of whether sqlite3 succeeded). Both functions now share one_ctx_sqlite_exec_atomichelper (also closing a reuse-review finding that the fix reinvented a pattern already established inmemory.sh'smemory_usage_commitrather than citing it), and_ctx_index_file's caller now treats its failure as fatal — unlike FTS, a failed primary-index write must not be reported as "re-indexed".Gate findings fixed (2 rounds)
content_ftsdidn't exist before the rebuild attempt (first-time build), the message wrongly said "existing (now stale) FTS index retained" — there was no existing index to retain. Fixed by branching the message on whethercontent_ftsexisted beforehand.Verification
pmctl gate run --executor codex --policy generic— Final: GO (critic/qa-tester/architecture-reviewer/security-reviewer, tierstandard, 5 rounds total — 2 real findings fixed, remaining rounds were gate-executor synthesis protocol instability unrelated to the code, diagnosed viasupervisor-stdout.log).tests/shell/test-pmctl-context.sh— 168 passed, 0 failed (7 new/strengthened cases covering: FTS rollback preserves the old index,_ctx_index_file's return code reflects sqlite3's real result, both callers report failure honestly in their stdout summary, and the first-time-build-failure wording).tests/bin/run-tests.sh --base main— 9/9 diff-selected suites passed (incl.lint-scripts).tests/bin/run-all-tests.sh— 104 passed, 0 failed, 0 skipped (full sign-off).🤖 Generated with Claude Code
https://claude.ai/code/session_0177ds9nxEMtsh3bwpBxNHYx