From cc53de52100de12dcb4348122c37ceae7032348d Mon Sep 17 00:00:00 2001 From: Lien Chen Date: Wed, 26 Aug 2026 17:28:19 +0900 Subject: [PATCH 1/3] test(lint-shellcheck): replace sleep-based worker-overlap proof with a file barrier test_default_worker_cap used sleep 0.1 inside the ShellCheck stub to widen a race window for observing two-way concurrency, which QA rules forbid and which host load can silently weaken (max_active could observe 1 even when the real cap is 2, since the old assertion only required >= 1). Workers now register at a flock-serialized counter and all release together once exactly two are concurrently registered, making the max_active == 2 assertion a deterministic property of the protocol. A bounded 5s deadline fails the stub with a clear message instead of hanging if the cap ever degrades to 1. Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_0177ds9nxEMtsh3bwpBxNHYx --- BACKLOG.md | 14 ++++++-- tests/shell/test-lint-shellcheck.sh | 52 ++++++++++++++++++++++++----- 2 files changed, 55 insertions(+), 11 deletions(-) diff --git a/BACKLOG.md b/BACKLOG.md index 5346877c..325a8d11 100644 --- a/BACKLOG.md +++ b/BACKLOG.md @@ -39,7 +39,7 @@ CC-001/CC-002 were consumed by PR #24 fix bundle inline, with no standalone entr | CC-562 | ✅ done | synthesis／reviewer 驗證器仍有多個「多約束共用單一 reason 字串」分支（`invalid coverage matrix`、`invalid finding inventory or union`、`duplicate finding ID collision`、`selected/not-reviewed dimensions mismatch`），單次修正重試收到後無法行動；[[CC-553]] Req 2 判定需同等精度但屬不同 helper 形狀（逐項指出違規條目，非集合差集），故分票 | ops/gate | 2026-08-19 | pr:#510 | P3 | hygiene | | CC-560 | ✅ done | `_gate_scope_reference_index_collect` 每筆 reference 都以 `jq -nc` 建一個 JSON 物件（實測 4.9s×2），與 [[CC-557]] 已修掉的 `_gate_scope_expansion_append` 是同一類寫法；CC-557 未一併處理是因預算餘裕已足，非因不成立 | ops/gate | 2026-08-19 | pr:#511 | P3 | hygiene | | CC-554 | 🔵 active | 永久 regression test 缺少准入門檻：`/ship` 規範「修完每個 finding」但不規範修法形式，reviewer 每提一個邊界就永久長一個阻擋 case，case 又需要 meta-test 保護；QA 規則加六條准入條件＋五條替代路徑，ship.md 加對應例外（明確不設輪數上限，見 [[CC-544]]） | ops/gate | 2026-08-17 | — | P1 | hygiene | -| CC-552 | 🔵 active | `test_default_worker_cap` 以 `sleep 0.1` 製造 worker 重疊窗口來驗證併發上限，違反 QA 規則的「不得以 sleep 同步」；主機負載會改變觀測到的重疊數，與 worker-cap 正確性無關（2026-08-17 CC-551 gate round 4 qa-tester，pre-existing） | ops/test | 2026-08-17 | — | P3 | hygiene | +| CC-552 | ✅ done | `test_default_worker_cap` 以 `sleep 0.1` 製造 worker 重疊窗口來驗證併發上限，違反 QA 規則的「不得以 sleep 同步」；主機負載會改變觀測到的重疊數，與 worker-cap 正確性無關（2026-08-17 CC-551 gate round 4 qa-tester，pre-existing） | ops/test | 2026-08-17 | — | P3 | hygiene | | CC-548 | ✅ closed 2026-08-26 | **[context.db FTS5 對 CJK 查詢無索引無排序]** Spike 判定 **AMBER — defer，暫不實作**：無 sqlite 版本下限硬衝突，rebuild 遷移成本為零，但實測 trigram rebuild 慢 ~4.9x、索引大 +32.6%，而品質增益在本 repo 實際語料上僅小幅（68 筆抽樣命中中多 1 筆），且原票「unicode61 無 ranking」前提經量測不成立。See `docs/spikes/CC-548.md`. | memory | 2026-08-16 | — | P2 | retrieval | | CC-466 | ⏸ deferred | 記憶卡片生命週期閉環：expires_at 執行 + 關窗式 supersede + usage sidecar 休眠偵測 + doctor→distill 接線；僅在 CC-467 證明 stale/dormant card 已形成實際問題時啟動 | memory | 2026-07-07 | feedback:2026-07-07 | P2 | retrieval | | CC-468 | ⏸ deferred | dispatch brief 帶 memory 約束與信任邊界；完成 CC-465→CC-467 後，僅在 usage evidence 證明有價值時啟動 | ops/memory | 2026-07-07 | — | P2 | retrieval | @@ -451,7 +451,7 @@ qa-tester／risk-reviewer 連擋並全數 revert。減量要從 finding 端做 --- -## CC-552 — worker-cap 測試以 sleep 製造重疊窗口 🔵 active +## CC-552 — worker-cap 測試以 sleep 製造重疊窗口 **Problem**: `tests/shell/test-lint-shellcheck.sh` 的 `test_default_worker_cap` 用 ShellCheck stub 內的 `sleep 0.1` 撐開一個時間窗，好讓兩個 worker 的 @@ -473,6 +473,16 @@ pre-existing 缺陷、與 ShellCheck 解析改動無關，且改法本身有風 **Cross-link**: [[CC-551]]（發現時點）、[[CC-543]]（bounded handshake 的既有教訓）。 +**Update 2026-08-26（done，pr:#pending）**: stub `shellcheck` 改用檔案式 barrier +（flock 序列化的計數器 + release sentinel），每個 worker 在起跑時先原子遞增計 +數並記錄觀測值，計數達 2 才建立 release、所有等待者才一起放行；斷言改為 +`max_active == 2`（原本的 `>= 1` 對「有沒有真的重疊」是無鑑別力的）。等待 +release 設 5 秒有界逾時，失敗訊息明確——若上限真的退化成 1，逾時後乾脆失敗而 +非無界等待，避開 [[CC-543]] 記錄過的 FIFO handshake hang 形狀（本設計不用 +FIFO，全程只靠一般檔案 + `flock`，因此不會撞上「parent 需在 fork 前對 FIFO 持 +有 O_RDWR」那個坑）。範圍如票面 Requirement 3 所限，未動 +`lint-shellcheck.sh` 本身的併發實作。 + --- ## CC-548 — context.db FTS5 對 CJK 查詢無索引無排序（spike）✅ 2026-08-26 diff --git a/tests/shell/test-lint-shellcheck.sh b/tests/shell/test-lint-shellcheck.sh index 5c6789f4..d5696ec5 100755 --- a/tests/shell/test-lint-shellcheck.sh +++ b/tests/shell/test-lint-shellcheck.sh @@ -533,13 +533,21 @@ test_worker_override_ceiling() { } # Behavior: the local linter does not turn host CPU count into unbounded parallelism. -# Steps: use an instrumented ShellCheck stub; assert the default never exceeds two workers. +# Steps: use an instrumented ShellCheck stub whose invocations hold at a shared +# file-based barrier until exactly two are concurrently registered, then all +# release together. This makes the two-way overlap a deterministic property of +# the barrier protocol rather than a hope that a fixed sleep window lines up +# with host scheduling. If the real cap ever collapses to one, no second +# worker ever registers, so the lone worker waits at the barrier until a +# bounded deadline expires and fails with a clear message instead of hanging. test_default_worker_cap() { - local name="lint-shellcheck/default-worker-cap" root events output status=0 max_active + local name="lint-shellcheck/default-worker-cap" root events barrier output status=0 max_active should_run "$name" || return 0 root="$(fixture_repo worker-cap)" events="$root/events.log" - mkdir -p "$root/bin" + barrier="$root/barrier" + mkdir -p "$root/bin" "$barrier" + printf '0' > "$barrier/count" for n in 1 2 3 4; do printf '#!/usr/bin/env bash\nset -euo pipefail\n' > "$root/tests/worker-$n.sh" done @@ -551,20 +559,46 @@ if [[ "${1:-}" == --version ]]; then exit 0 fi events="${SHELLCHECK_EVENTS:?}" -printf 'start %s\n' "$$" >> "$events" -sleep 0.1 -printf 'end %s\n' "$$" >> "$events" +barrier="${SHELLCHECK_BARRIER_DIR:?}" +lock="$barrier/lock" +counter="$barrier/count" +release="$barrier/release" + +exec {lockfd}>"$lock" +flock "$lockfd" +count=$(<"$counter") +count=$((count + 1)) +printf '%s' "$count" > "$counter" +printf 'active %s\n' "$count" >> "$events" +[[ "$count" -lt 2 ]] || : > "$release" +flock -u "$lockfd" + +deadline=$((SECONDS + 5)) +until [[ -e "$release" ]]; do + if [[ "$SECONDS" -ge "$deadline" ]]; then + printf 'shellcheck-stub: timed out waiting for a second concurrent worker\n' >&2 + exit 1 + fi + sleep 0.02 +done + +exec {lockfd}>"$lock" +flock "$lockfd" +count=$(<"$counter") +count=$((count - 1)) +printf '%s' "$count" > "$counter" +flock -u "$lockfd" STUB chmod +x "$root/bin/shellcheck" # Assert the built-in default, not an override inherited from the caller. output="$(env -u PM_DISPATCH_SHELLCHECK_JOBS PATH="$root/bin:$PATH" SHELLCHECK_EVENTS="$events" \ + SHELLCHECK_BARRIER_DIR="$barrier" \ bash "$root/tools/lint/lint-shellcheck.sh" --repo "$root" 2>&1)" || status=$? max_active="$(awk ' - $1 == "start" { active++; if (active > max) max = active } - $1 == "end" { active-- } + $1 == "active" && $2 > max { max = $2 } END { print max + 0 } ' "$events")" - if [[ "$status" -eq 0 && "$max_active" -le 2 && "$max_active" -ge 1 ]]; then + if [[ "$status" -eq 0 && "$max_active" -eq 2 ]]; then pass "$name" else fail "$name" "status=$status max_active=$max_active output=$output" From 0eb31a94aa9517acd88b04cb59c0c5ab3d435a17 Mon Sep 17 00:00:00 2001 From: Lien Chen Date: Wed, 26 Aug 2026 17:46:30 +0900 Subject: [PATCH 2/3] fixup: replace remaining barrier sleep with a blocking FIFO wait qa-tester's gate review correctly flagged that the bounded busy-wait (sleep-in-a-loop) still counted as forbidden sleep-based synchronization. Switch to a blocking read on a bidirectionally-opened FIFO (open O_RDWR never blocks, sidestepping the open()-hangs-forever failure mode a prior ticket hit with a read-only/write-only FIFO) with a bounded -t timeout as the only failure path, and reuse the repo's existing serialize_with_lock helper for the counter mutex instead of a bespoke flock call (critic's portability finding). Also fixed the fixture's total shell-file count, which was odd and left the last worker of the sliding window unpaired. Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_0177ds9nxEMtsh3bwpBxNHYx --- BACKLOG.md | 19 ++++--- tests/shell/test-lint-shellcheck.sh | 88 +++++++++++++++++------------ 2 files changed, 61 insertions(+), 46 deletions(-) diff --git a/BACKLOG.md b/BACKLOG.md index 325a8d11..43e7a958 100644 --- a/BACKLOG.md +++ b/BACKLOG.md @@ -473,15 +473,16 @@ pre-existing 缺陷、與 ShellCheck 解析改動無關，且改法本身有風 **Cross-link**: [[CC-551]]（發現時點）、[[CC-543]]（bounded handshake 的既有教訓）。 -**Update 2026-08-26（done，pr:#pending）**: stub `shellcheck` 改用檔案式 barrier -（flock 序列化的計數器 + release sentinel），每個 worker 在起跑時先原子遞增計 -數並記錄觀測值，計數達 2 才建立 release、所有等待者才一起放行；斷言改為 -`max_active == 2`（原本的 `>= 1` 對「有沒有真的重疊」是無鑑別力的）。等待 -release 設 5 秒有界逾時，失敗訊息明確——若上限真的退化成 1，逾時後乾脆失敗而 -非無界等待，避開 [[CC-543]] 記錄過的 FIFO handshake hang 形狀（本設計不用 -FIFO，全程只靠一般檔案 + `flock`，因此不會撞上「parent 需在 fork 前對 FIFO 持 -有 O_RDWR」那個坑）。範圍如票面 Requirement 3 所限，未動 -`lint-shellcheck.sh` 本身的併發實作。 +**Update 2026-08-26（done，pr:#pending）**: stub `shellcheck` 改用事件式 +barrier：每個 worker 起跑時先透過既有的 `serialize_with_lock`（沿用 +production 本來就在用的可攜式鎖，而非另外重造一個）原子遞增計數並記錄觀測值； +第一個抵達的 worker 對一個雙向開啟（`<>`，避免只用唯讀/唯寫端造成 open() +本身卡住——這正是 [[CC-543]] 記錄過的 FIFO handshake hang 形狀）的 FIFO 做 +有界（5 秒逾時）阻塞式 `read`，第二個抵達的 worker 寫入一行將其釋放，兩者才 +一起往下跑；全程沒有任何 `sleep`。斷言也從 `max_active >= 1` 收緊為 +`max_active == 2`（原本的門檻對「有沒有真的重疊」其實是無鑑別力的）。若上限 +真的退化成 1，第二個 worker 永遠不會出現，逾時後乾脆失敗且訊息明確，而非無界 +等待。範圍如票面 Requirement 3 所限，未動 `lint-shellcheck.sh` 本身的併發實作。 --- diff --git a/tests/shell/test-lint-shellcheck.sh b/tests/shell/test-lint-shellcheck.sh index d5696ec5..ae9dfa52 100755 --- a/tests/shell/test-lint-shellcheck.sh +++ b/tests/shell/test-lint-shellcheck.sh @@ -533,13 +533,16 @@ test_worker_override_ceiling() { } # Behavior: the local linter does not turn host CPU count into unbounded parallelism. -# Steps: use an instrumented ShellCheck stub whose invocations hold at a shared -# file-based barrier until exactly two are concurrently registered, then all -# release together. This makes the two-way overlap a deterministic property of -# the barrier protocol rather than a hope that a fixed sleep window lines up -# with host scheduling. If the real cap ever collapses to one, no second -# worker ever registers, so the lone worker waits at the barrier until a -# bounded deadline expires and fails with a clear message instead of hanging. +# Steps: use an instrumented ShellCheck stub whose invocations register at a +# shared counter (guarded by the repo's own serialize_with_lock, so this stays +# on the same portable-locking primitive production code already depends on) +# and block on a blocking FIFO read -- never a sleep loop -- until a second +# concurrent worker signals release. This makes the two-way overlap a +# deterministic property of the barrier protocol rather than a hope that a +# fixed sleep window lines up with host scheduling. If the real cap ever +# collapses to one, no second worker ever arrives to signal release, so the +# lone worker's bounded FIFO read times out and fails with a clear message +# instead of hanging. test_default_worker_cap() { local name="lint-shellcheck/default-worker-cap" root events barrier output status=0 max_active should_run "$name" || return 0 @@ -548,46 +551,57 @@ test_default_worker_cap() { barrier="$root/barrier" mkdir -p "$root/bin" "$barrier" printf '0' > "$barrier/count" - for n in 1 2 3 4; do + mkfifo "$barrier/release.fifo" + # The barrier pairs workers strictly by arrival order, so the fixture's + # total shell-file count (these five plus the three fixture_repo already + # creates) must be even -- an odd file out would have no partner to + # release it. + for n in 1 2 3 4 5; do printf '#!/usr/bin/env bash\nset -euo pipefail\n' > "$root/tests/worker-$n.sh" done - cat > "$root/bin/shellcheck" <<'STUB' + cat > "$root/bin/shellcheck" <"$lock" -flock "$lockfd" -count=$(<"$counter") -count=$((count + 1)) -printf '%s' "$count" > "$counter" -printf 'active %s\n' "$count" >> "$events" -[[ "$count" -lt 2 ]] || : > "$release" -flock -u "$lockfd" - -deadline=$((SECONDS + 5)) -until [[ -e "$release" ]]; do - if [[ "$SECONDS" -ge "$deadline" ]]; then +# shellcheck source=/dev/null +. "$REPO_ROOT/runtime/lib/portable.sh" +events="\${SHELLCHECK_EVENTS:?}" +barrier="\${SHELLCHECK_BARRIER_DIR:?}" +counter="\$barrier/count" +mine="\$barrier/count.\$\$" + +register_arrival() { + local count + count=\$(<"\$counter") + count=\$((count + 1)) + printf '%s' "\$count" > "\$counter" + printf 'active %s\n' "\$count" >> "\$events" + printf '%s' "\$count" > "\$mine" +} +serialize_with_lock "\$barrier/lock" register_arrival +count="\$(<"\$mine")" +rm -f "\$mine" + +exec {relfd}<>"\$barrier/release.fifo" +if [[ "\$count" -eq 1 ]]; then + if ! read -r -t 5 -u "\$relfd" _; then printf 'shellcheck-stub: timed out waiting for a second concurrent worker\n' >&2 exit 1 fi - sleep 0.02 -done - -exec {lockfd}>"$lock" -flock "$lockfd" -count=$(<"$counter") -count=$((count - 1)) -printf '%s' "$count" > "$counter" -flock -u "$lockfd" +else + printf '\n' >&"\$relfd" +fi + +deregister() { + local count + count=\$(<"\$counter") + count=\$((count - 1)) + printf '%s' "\$count" > "\$counter" +} +serialize_with_lock "\$barrier/lock" deregister STUB chmod +x "$root/bin/shellcheck" # Assert the built-in default, not an override inherited from the caller. From c283ec6a3861204564a9a3c1f1323a17ccee1aaf Mon Sep 17 00:00:00 2001 From: Lien Chen Date: Wed, 26 Aug 2026 18:05:49 +0900 Subject: [PATCH 3/3] fixup: add ack handshake so barrier release does not exit before partner wakes qa-tester's second gate round found that the releasing worker wrote its release signal and immediately proceeded toward deregistration without confirming the released worker actually woke up -- under a hypothetical cap-3 regression, both original workers could deregister before a concurrently-forked third worker reached the counter lock, letting the observed max_active read 2 even though 3 were briefly resident. Add a second FIFO for a blocking ack: the releaser now waits for the released worker's ack before either side deregisters, closing that asymmetric fast-exit window. Verified with a jobs=3 mutation of the fixture's copy of lint-shellcheck.sh -- the test now observes max_active=6 and fails, where it previously would have reported 2 and passed. Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_0177ds9nxEMtsh3bwpBxNHYx --- BACKLOG.md | 8 +++++++- tests/shell/test-lint-shellcheck.sh | 13 ++++++++++++- 2 files changed, 19 insertions(+), 2 deletions(-) diff --git a/BACKLOG.md b/BACKLOG.md index 43e7a958..7f4ecfb6 100644 --- a/BACKLOG.md +++ b/BACKLOG.md @@ -482,7 +482,13 @@ production 本來就在用的可攜式鎖，而非另外重造一個）原子遞 一起往下跑；全程沒有任何 `sleep`。斷言也從 `max_active >= 1` 收緊為 `max_active == 2`（原本的門檻對「有沒有真的重疊」其實是無鑑別力的）。若上限 真的退化成 1，第二個 worker 永遠不會出現，逾時後乾脆失敗且訊息明確，而非無界 -等待。範圍如票面 Requirement 3 所限，未動 `lint-shellcheck.sh` 本身的併發實作。 +等待。gate 第二輪 qa-tester 進一步指出：release 的一方寫完信號就立刻繼續走向 +deregister，並未等對方真的醒來，若上限退化成 3，前兩個 worker 有機會在第三個 +worker 搶到鎖之前就雙雙 deregister，讓事件記錄看起來仍是 2；修法是加第二個 +FIFO 做 ack 交握——release 方在放行後阻塞讀 ack，等對方真的從 barrier 醒來並 +回覆才繼續，兩邊因此保證同時退場，不會有一方搶先鬆手。已用 `jobs` 改 3 的 +mutation 直接驗證：max_active 從 2 變成可觀測到 6，測試如預期紅燈。範圍如票面 +Requirement 3 所限，未動 `lint-shellcheck.sh` 本身的併發實作。 --- diff --git a/tests/shell/test-lint-shellcheck.sh b/tests/shell/test-lint-shellcheck.sh index ae9dfa52..7aa9ab6c 100755 --- a/tests/shell/test-lint-shellcheck.sh +++ b/tests/shell/test-lint-shellcheck.sh @@ -552,6 +552,7 @@ test_default_worker_cap() { mkdir -p "$root/bin" "$barrier" printf '0' > "$barrier/count" mkfifo "$barrier/release.fifo" + mkfifo "$barrier/ack.fifo" # The barrier pairs workers strictly by arrival order, so the fixture's # total shell-file count (these five plus the three fixture_repo already # creates) must be even -- an odd file out would have no partner to @@ -585,14 +586,24 @@ serialize_with_lock "\$barrier/lock" register_arrival count="\$(<"\$mine")" rm -f "\$mine" -exec {relfd}<>"\$barrier/release.fifo" +exec {relfd}<>"\$barrier/release.fifo" {ackfd}<>"\$barrier/ack.fifo" if [[ "\$count" -eq 1 ]]; then if ! read -r -t 5 -u "\$relfd" _; then printf 'shellcheck-stub: timed out waiting for a second concurrent worker\n' >&2 exit 1 fi + printf '\n' >&"\$ackfd" else printf '\n' >&"\$relfd" + # Wait for the released worker's own ack before either side deregisters -- + # without this, the releaser can decrement and exit before the released + # worker's read has actually returned, closing the overlap window before a + # genuinely-concurrent third worker (were the cap to regress) has a chance + # to register while both are still counted active. + if ! read -r -t 5 -u "\$ackfd" _; then + printf 'shellcheck-stub: timed out waiting for released worker ack\n' >&2 + exit 1 + fi fi deregister() {