fix(CC-571): sqlite atomic-script gap in _ctx_fts_rebuild + _ctx_index_file

BACKLOG.md

@@ -107,6 +107,7 @@ CC-001/CC-002 were consumed by PR #24 fix bundle inline, with no standalone entr
 | CC-568 | 🟢 someday | `/mem-distill` Case→Strategy 機械式提升：對 `episodes.jsonl` 既有結構化欄位做 count/cluster 門檻判定，取代逐次主觀「感覺像 pattern」的判斷；依賴 [[CC-567]] 的 outcome 證據決定是否值得做（2026-08-25 memory 架構設計討論） | memory/DX | 2026-08-25 | — | P2 | retrieval |
 | CC-569 | 🟢 someday | `pmctl task` / `context pack` 擴充 working-memory 敘事欄位（`selected_memories`／`rejected_paths`／`blockers`／`next_action`）：延伸既有 schema，不新建第二個「現在在幹嘛」真相來源；依賴 [[CC-567]] 證明有價值後再排（2026-08-25 memory 架構設計討論） | memory/DX | 2026-08-25 | — | P2 | design |
 | CC-570 | 🟢 someday | Fact/Case/Strategy `memory_function`／`memory_subtype` metadata 分類法：先蒐集 [[CC-567]] 的 applied/outcome 證據，再決定值不值得建分類機制——不憑直覺先建立稅務式標籤（2026-08-25 memory 架構設計討論；外部文章優先序建議相反，本 repo 刻意反過來） | memory/DX | 2026-08-25 | — | P3 | retrieval |
+| CC-571 | 🔵 active | `_ctx_fts_rebuild`／`_ctx_index_file` 共用的 sqlite atomic-script 缺口：DROP+CREATE+INSERT 未加 `-bail`（實測 sqlite3 CLI 預設不會在錯誤時中止，單靠 BEGIN/COMMIT 不足）、呼叫端不檢查回傳值、`_ctx_index_file` 還有第三個獨立 bug（`rm -f` 蓋掉 sqlite3 真實 exit code）；`/simplify` altitude review 抓到手足函式同缺陷，範圍已擴大涵蓋兩者（[[CC-548]] spike 的 Open risks 側面發現，非本票 tokenizer 範圍） | memory/ops | 2026-08-26 | — | P2 | hygiene |
 
 ---
 
@@ -3333,3 +3334,63 @@ machinery，是憑一篇文章的直覺蓋機制，屬於本 repo 已經吃過
 **Dependencies**: 前置 = [[CC-567]] shipped + 觀察窗證據。P3，不預設排入 milestone。
 
 ---
+
+## CC-571 — sqlite atomic-script 缺口：`_ctx_fts_rebuild`／`_ctx_index_file` 🔵 active
+
+**Problem**: `runtime/lib/pmctl-context.sh` 的 `_ctx_fts_rebuild()` 對
+`content_fts` 做 `DROP TABLE` → `CREATE VIRTUAL TABLE` → 兩個 `INSERT ... SELECT`，
+整段用 heredoc 餵給 `sqlite3 "$db" >/dev/null`，沒有 `BEGIN`/`COMMIT`。兩個呼叫端
+（`pmctl_context_index` 約 line 795、`pmctl_context_update` 約 line 1126）都是裸呼叫
+`_ctx_fts_rebuild "$db"`，不檢查回傳值，之後照樣印「context index/update」成功訊息。
+
+**Why**: 直接實測證實這不是理論風險。用一個蓄意中途出錯的重建腳本測試：
+1. 不加 `-bail`：sqlite3 CLI 預設遇到錯誤只印訊息、**不中止**，照樣跑到 `COMMIT`
+   （若有包 transaction 也一樣會提交半成功的內容）；exit code 雖然是 1，但呼叫端
+   從不檢查。
+2. 加 `-bail` 後才會在第一個錯誤處真正中止，交易維持未提交，行程結束時連線關閉
+   觸發自動 rollback，舊的 `content_fts` 完整保留（已用最小 repro 驗證）。
+
+三個問題疊在一起：(a) 沒有 atomicity——失敗可能留下半建或整個消失的表；(b) 沒有
+`-bail`，單靠 `BEGIN`/`COMMIT` 不足以達成 (a) 的保護；(c) 呼叫端不檢查回傳值，
+即使 (a)(b) 都修好，使用者也不會知道索引其實是舊的（rollback 後）卻顯示重建成功。
+本票是 [[CC-548]] spike 過程中在 Open risks 側面發現的既有缺口，與該票的 tokenizer
+判斷（AMBER，暫緩）完全無關；使用者已明確要求只處理這個 bug，不連動 trigram 切換。
+
+**Requirement**:
+1. `_ctx_fts_rebuild` 的 DROP/CREATE/INSERT 序列包進單一交易（`BEGIN
+   IMMEDIATE`…`COMMIT`），並對 `sqlite3` 呼叫加 `-bail`（或等效機制），確保任何一步
+   出錯都會在該步中止、交易不提交，使既有 `content_fts` 保持完整可查詢，而不是
+   半建或消失。
+2. `_ctx_fts_rebuild` 的失敗必須讓呼叫端可辨——回傳非零，且兩個呼叫端
+   （`pmctl_context_index`／`pmctl_context_update`）改為檢查其回傳值：失敗時不得
+   印「成功」字樣的訊息，改為誠實回報「FTS 索引重建失敗，仍使用既有索引」一類的
+   降級狀態（比照本 repo既有 `usage_store: error`／`resolution_issues` 誠實回報慣例，
+   不阻斷整體 index/update 流程——FTS 只是加速層，非唯一查詢路徑，LIKE fallback
+   仍可用）。
+3. Regression fixtures：模擬重建腳本中途失敗（例如注入一個會觸發 SQL 錯誤的條件），
+   斷言 (a) 舊 `content_fts` 內容不變、(b) `_ctx_fts_rebuild` 回傳非零、(c) 呼叫端
+   印出的訊息誠實反映失敗、不宣稱成功。
+
+**Non-goals**: 不改 FTS5 tokenizer（unicode61 維持不變，[[CC-548]] 已判 AMBER 暫緩）；
+不新增 schema 欄位或 `index_meta` 版本追蹤；不處理 query-during-rebuild 的
+讀者可見性問題本身（rollback 後舊表持續可查詢，交易保護已隱含解決多數場景）。
+
+**Update 2026-08-26（範圍擴大，實作中）**：`/simplify` 的 altitude review 在同一輪
+reuse/簡化確認裡抓到手足函式同缺陷——`_ctx_index_file()`（`pmctl_context_update`
+另一個呼叫路徑，寫的是 files／symbols／file_chunks 主索引資料，非 FTS 加速層）用
+`BEGIN;`…`COMMIT;` 但同樣沒加 `-bail`；直接測試還額外找到第三個獨立 bug：其函式
+本體最後一行是 `sqlite3 ...; rm -f "$tmpf"`，函式回傳值變成 `rm` 的 exit code（幾乎
+恆為 0），完全蓋掉 sqlite3 真正的失敗狀態，即使先前已加 `-bail` 也測不出來。範圍
+擴大為：兩個函式共用同一個新抽出的 `_ctx_sqlite_exec_atomic` helper（單一
+`-bail` 呼叫來源，同時解決 reuse review 指出的「兩處各自重新推導同一手法」）；
+`_ctx_index_file` 明確 `return "$rc"`（在 `rm` 之前先擷取），且其唯一呼叫端
+（`pmctl_context_update`）失敗時視為**致命**（不同於 FTS——這是主索引資料而非
+best-effort 加速層，宣稱「re-indexed」等於說謊）。新增對應 regression fixtures
+（`_ctx_index_file` 回傳碼、`pmctl_context_update` 失敗時不宣稱成功）。使用者已
+確認此擴大屬於「同一個 bug」範圍內的自然延伸，非另開新工。
+
+**Cross-link**: [[CC-548]]（spike 中發現本缺口，Open risks 段落）。也可見
+`runtime/lib/memory.sh` 的 `memory_usage_commit`（既有的 `-bail` atomic-script
+先例，本票的 helper 命名與理由都直接引用它，而非各自重新推導）。
+
+---

runtime/lib/pmctl-context.sh

@@ -602,20 +602,53 @@ _ctx_generate_file_sql() {
   done < <(_ctx_chunk_file "$abs_path" "$lang")
 }
 
+# CC-571: shared atomic-script executor for this file's two DROP/CREATE/
+# INSERT-style rebuild scripts (_ctx_index_file, _ctx_fts_rebuild). Runs the
+# caller-supplied SQL (via stdin, which the caller wraps in its own
+# BEGIN IMMEDIATE/COMMIT) through `sqlite3 -bail`.
+#
+# `-bail` is not optional decoration: confirmed by direct reproduction, the
+# sqlite3 CLI's default behavior on a mid-script SQL error is to print the
+# error and keep executing subsequent statements (it does NOT stop) -- so a
+# bare BEGIN...COMMIT without -bail still reaches and executes COMMIT after
+# silently skipping the failed statement, committing a half-built table.
+# With -bail, an error aborts the script immediately, the transaction is
+# left open, and the sqlite3 process exiting closes the connection, which
+# triggers an automatic ROLLBACK -- prior committed state is left fully
+# intact. This mirrors the `-bail`-based atomic-script pattern already
+# established in memory.sh's memory_usage_commit (~line 404) for the same
+# reason; that instance also captures stderr and retries on a lock error,
+# which neither caller here currently needs (both run at most once per
+# `context index`/`context update` invocation, not under contention).
+#
+# Returns sqlite3's exit code. Callers MUST check it -- this function does
+# not decide whether a failure here is fatal to the caller; that differs
+# per call site (see CC-571 Requirement 2).
+_ctx_sqlite_exec_atomic() {
+  local db="$1"
+  sqlite3 -bail "$db" >/dev/null
+}
+
 # ── Single-file index (used by pmctl_context_update) ──────────────────────────
 
 _ctx_index_file() {
   local db="$1" abs_path="$2" rel_path="$3"
-  local tmpf
+  local tmpf rc
   tmpf="$(mktemp /tmp/ctx-XXXXXX.sql)"
   {
     printf 'PRAGMA busy_timeout=5000;\n'
-    printf 'BEGIN;\n'
+    printf 'BEGIN IMMEDIATE;\n'
     _ctx_generate_file_sql "$abs_path" "$rel_path"
     printf 'COMMIT;\n'
   } > "$tmpf"
-  sqlite3 "$db" < "$tmpf" >/dev/null
+  _ctx_sqlite_exec_atomic "$db" < "$tmpf"
+  rc=$?
   rm -f "$tmpf"
+  # rc must be captured before `rm` -- rm's own exit status would otherwise
+  # become this function's return value regardless of whether sqlite3
+  # actually succeeded (confirmed by direct reproduction: the original
+  # `sqlite3 ... ; rm -f "$tmpf"` shape always returned 0).
+  return "$rc"
 }
 
 # ── FTS5 index rebuild ─────────────────────────────────────────────────────────
@@ -628,8 +661,9 @@ _ctx_fts_rebuild() {
   # each hit's real bounded span instead of faking line_end=line_start (the
   # pack/query contract advertises line_start/line_end as the actual chunk or
   # symbol extent -- CC-505 Req 3 gate finding critic-F001).
-  sqlite3 "$db" >/dev/null <<'SQLFTS'
+  _ctx_sqlite_exec_atomic "$db" <<'SQLFTS'
 PRAGMA busy_timeout=5000;
+BEGIN IMMEDIATE;
 DROP TABLE IF EXISTS content_fts;
 CREATE VIRTUAL TABLE content_fts USING fts5(ref, text, line_end UNINDEXED);
 INSERT INTO content_fts(ref, text, line_end)
@@ -639,6 +673,7 @@ INSERT INTO content_fts(ref, text, line_end)
   SELECT f.path || ':' || fc.line_start, TRIM(COALESCE(fc.heading, '') || ' ' || COALESCE(fc.text, '')), fc.line_end
   FROM file_chunks fc JOIN files f ON fc.file_id = f.id
   WHERE TRIM(COALESCE(fc.heading, '') || ' ' || COALESCE(fc.text, '')) != '';
+COMMIT;
 SQLFTS
 }
 
@@ -790,12 +825,38 @@ _ctx_index_tree() {
   # Rebuild only for changed files or a path-count change (pure deletions).
   local _fts_present
   _fts_present="$(sqlite3 "$db" "SELECT count(*) FROM sqlite_master WHERE type='table' AND name='content_fts';" 2>/dev/null || printf '0')"
+  # CC-571: FTS rebuild is a best-effort acceleration layer, not the only
+  # query path (LIKE fallback remains available), so a failed rebuild does
+  # not fail the overall index -- but neither the stderr diagnostic NOR the
+  # final stdout summary may read as an unqualified success (gate finding
+  # critic-F001, round 1: printing the diagnostic on stderr while stdout
+  # still said a bare "N indexed, M skipped" is a contradictory summary --
+  # a caller that only looks at stdout, or at exit code 0, saw only
+  # success). The previous content_fts (rolled back to, not left
+  # half-built -- see _ctx_fts_rebuild) is now stale relative to the
+  # indexed content above.
+  local _fts_rebuild_note=""
   if (( indexed > 0 || found != ${#_ctx_db_mtimes[@]} )) || [[ "$_fts_present" != "1" ]] \
      || [[ "$_force_reextract" -eq 1 ]]; then
-    _ctx_fts_rebuild "$db"
+    if ! _ctx_fts_rebuild "$db"; then
+      # CC-571 gate finding critic-F001 (round 2, gate-20260826-021038-ac0bc2):
+      # when content_fts did not exist before this attempt (first-time
+      # build), a failed rebuild's rollback leaves NO FTS table at all --
+      # "existing (now stale) FTS index retained" would be false in that
+      # case, since there is no existing index to retain. _fts_present was
+      # captured before the rebuild attempt, so it still reflects the
+      # pre-rebuild state here.
+      if [[ "$_fts_present" == "1" ]]; then
+        printf 'pmctl context index: FTS index rebuild failed; existing (now stale) FTS index retained, LIKE fallback still available\n' >&2
+      else
+        printf 'pmctl context index: FTS index rebuild failed; no FTS index available, LIKE fallback only\n' >&2
+      fi
+      _fts_rebuild_note=' (FTS index degraded: rebuild failed, stale index retained)'
+      [[ "$_fts_present" == "1" ]] || _fts_rebuild_note=' (FTS index degraded: rebuild failed, no FTS index available)'
+    fi
   fi
 
-  printf 'context index: %d indexed, %d skipped\n' "$indexed" "$skipped"
+  printf 'context index: %d indexed, %d skipped%s\n' "$indexed" "$skipped" "$_fts_rebuild_note"
   printf 'db: %s\n' "$db"
 }
 
@@ -1122,9 +1183,33 @@ pmctl_context_update() {
       return 1
     fi
     local rel_path="${real_path#"$canon_root/"}"
-    _ctx_index_file "$db" "$real_path" "$rel_path"
-    _ctx_fts_rebuild "$db"
-    printf 'context update: re-indexed %s\n' "$rel_path"
+    # CC-571: unlike the FTS rebuild below, a failed _ctx_index_file is
+    # fatal to this command -- it writes the primary files/symbols/
+    # file_chunks data, not a best-effort acceleration layer, so a rollback
+    # here means the file's content is genuinely not reflected. Reporting
+    # "re-indexed" anyway would be a false claim, not a degraded state.
+    if ! _ctx_index_file "$db" "$real_path" "$rel_path"; then
+      printf 'pmctl context update: failed to index %s; index not updated for this file\n' "$rel_path" >&2
+      return 1
+    fi
+    # CC-571: see the matching comment in pmctl_context_index -- neither the
+    # stderr diagnostic nor the final stdout summary may read as an
+    # unqualified success when the FTS rebuild fails (gate finding
+    # critic-F001, round 1), and the message must not claim a stale index
+    # is "retained" when no content_fts existed before this attempt
+    # (gate finding critic-F001, round 2).
+    local _fts_rebuild_note="" _fts_present_before_update
+    _fts_present_before_update="$(sqlite3 "$db" "SELECT count(*) FROM sqlite_master WHERE type='table' AND name='content_fts';" 2>/dev/null || printf '0')"
+    if ! _ctx_fts_rebuild "$db"; then
+      if [[ "$_fts_present_before_update" == "1" ]]; then
+        printf 'pmctl context update: FTS index rebuild failed; existing (now stale) FTS index retained, LIKE fallback still available\n' >&2
+        _fts_rebuild_note=' (FTS index degraded: rebuild failed, stale index retained)'
+      else
+        printf 'pmctl context update: FTS index rebuild failed; no FTS index available, LIKE fallback only\n' >&2
+        _fts_rebuild_note=' (FTS index degraded: rebuild failed, no FTS index available)'
+      fi
+    fi
+    printf 'context update: re-indexed %s%s\n' "$rel_path" "$_fts_rebuild_note"
   else
     # No path given: full incremental scan (same as index with mtime check)
     pmctl_context_index "$repo_root"