fix: make Columnar deletes durable across reopen (flush-time compaction) - #365
Merged
Conversation
Columnar deletes were logical only (PK/hash-index removal), so the on-load PK-index rebuild resurrected deleted rows from the untouched .dat after a reopen. Logically deleted rows are now tracked (_pendingLogicalDeletes, incremented by DeleteRecordsCore and the contiguous bulk-delete fast path) and physically compacted at Table.Flush/Dispose via CompactPendingDeletes (Columnar + PK, outside a transaction). Regression test DeletesPersistAcrossReopen_AfterFlushCompaction deletes half the rows, flushes, reopens and asserts exactly the remaining rows come back.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Columnar (append-only) deletes were logical only —
DeleteRecordsCore/the contiguous bulk fast path removed PK/hash-index entries but never touched the.dat. Because the on-load PK-index rebuild reads every physical record, deleted rows came back after a reopen (repro: insert 100, delete 50,Flush, dispose, reopen → 100 rows again).Fix
Track logically-deleted rows (
Table._pendingLogicalDeletes, incremented by every Columnar delete path incl. the B9 contiguous bulk fast path) and physically compact them atTable.Flush/Disposevia the newCompactPendingDeletes()— Columnar tables with a PK, outside a transaction (deferred otherwise). Compaction rewrites the file keeping only live PK positions and rebuilds the indexes, so DELETE now survives reopen.Tests
Full suite 1656/1656. New regression
DeletesPersistAcrossReopen_AfterFlushCompaction: delete half the rows, flush, reopen → exactly the remaining rows come back (scan + COUNT). Existing delete/bulk tests (incl. scan/COUNT-after-delete assertions) stay green.