Skip to content

Stabilize transaction-log purge blast-radius coverage - #2244

Open
kriszyp wants to merge 2 commits into
mainfrom
fix/txnlog-purge-stale-read-blast
Open

Stabilize transaction-log purge blast-radius coverage#2244
kriszyp wants to merge 2 commits into
mainfrom
fix/txnlog-purge-stale-read-blast

Conversation

@kriszyp

@kriszyp kriszyp commented Aug 20, 2026

Copy link
Copy Markdown
Member

Summary

Stabilizes QA-782 without weakening its RocksDB stress arm. The apparent lost KEEP/CTRL reads were test-cascade failures: on failing Windows jobs, LMDB seeding timed out before those records existed; completed seeds read correctly on every covered surface, and the installed/locked RocksDB binding is 2.7.1 rather than the older affected release.

  • keep the RocksDB subject at 6,000 deleted plus 6,000 retained rows and require more than 16 MiB of transaction-log data
  • reduce only the LMDB control to 600 rows per range, since it does not exercise RocksDB transaction-log rotation
  • fail later phases closed unless the complete seed, control row, flush sequence, and disk preconditions succeed
  • match only the target ID's exact pre-cutoff insert audit entry
  • probe the deleted-record subject before exercising a known-present control across all five absence detectors

This is a test defect, not a product data-loss defect. Main failures show the seed phase timing out first, followed by requests for keys that phase never created; a successful main job with completed seeds passes the same ordinary-read checks for both engines.

Evidence

Verification

  • npm run build

  • seven targeted QA-782 runs during development, each 14/14 across RocksDB and LMDB; RocksDB continued to produce 21.182 MiB of .txnlog data

  • deliberate oracle mutant: a known-present ctrl0 was temporarily routed through the deleted-record probe; both engine arms failed in phase 4 (12/14), then the mutation was removed and the suite returned to 14/14

  • npm run lint:required

  • npm run format:write (no changes)

  • full integration run reached QA-782 under concurrent suite load, but the aggregate command remained red on unrelated real-Ollama tests

  • local unit aggregates are not authoritative on this host: test:unit:main initializes an installed system database before test path overrides, while test:unit:resources exposed installed config/state differences

  • PR-triggered run 32370864136: all six Windows Node 24 shards passed. Shard 1 ran both QA-782 arms successfully; RocksDB produced 37.177 MiB of .txnlog data and LMDB seeded in 8.4 seconds. The aggregate workflow's only failure was the adjacent QA-431 lost-increment assertion on uWS shard 3.

  • Repeated run 32370882792: the entire workflow passed, including all six Windows Node 24 shards. Shard 1 again ran both QA-782 arms successfully; RocksDB produced 37.177 MiB and LMDB seeded in 12.2 seconds.

Human reviewer decision

Please confirm the engine-specific volume choice: LMDB uses 600 rows per range while RocksDB retains 6,000. Equal volume would restore parity, but it also restores the Windows timeout that caused the false cascade and does not strengthen the RocksDB-only rotation invariant. This is reversible by changing one constant.

Co-Authored-By: GPT-5 Codex noreply@openai.com

Review-Coverage: authored=codex; ran=gemini; adjudicated=domain; declined=claude,cursor-grok,cursor-composer; rounds=3 @ 7d8c58c

Human-Review-Need: 3 (decisions: lmdb-reduced-volume) @ 7d8c58c

kriszyp and others added 2 commits August 20, 2026 06:33
Keep the RocksDB stress volume intact while right-sizing the LMDB control arm, make seed failures explicit, repair the audit cutoff matcher, and prove all five absence probes with a known-present control.

Co-Authored-By: GPT-5 Codex <noreply@openai.com>
Capture deleted-record results before the positive control's full scan can alter a first-read cache state, and keep the test comments focused on durable invariants.

Co-Authored-By: GPT-5 Codex <noreply@openai.com>
@kriszyp
kriszyp requested a review from cb1kenobi August 20, 2026 12:49

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors the stale-read blast radius integration test to dynamically scale the test volume based on the storage engine, reducing the workload size for LMDB while maintaining the larger stress volume for RocksDB. It introduces a helper function isPreCutoffInsert to refine audit log filtering, adds assertions to ensure the seeding phase completes before subsequent probes, and implements a positive control check to validate the absence detector. No review comments were provided, so there is no additional feedback.

@claude

claude Bot commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

Reviewed; no blockers found.

@kriszyp
kriszyp marked this pull request as ready for review August 20, 2026 19:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant