Skip to content

fix(storage): reopen data-integrity fixes for 2.0.0.2 - #390

Merged
MPCoreDeveloper merged 1 commit into
masterfrom
fix/reopen-roundtrip-data-integrity
Sep 4, 2026
Merged

fix(storage): reopen data-integrity fixes for 2.0.0.2#390
MPCoreDeveloper merged 1 commit into
masterfrom
fix/reopen-roundtrip-data-integrity

Conversation

@MPCoreDeveloper

Copy link
Copy Markdown
Owner

What

Three data-integrity fixes found by a reopen round-trip hardening sweep (all silent-data-loss class: valid writer output treated as end-of-data / resurrected on reopen):

  1. Directory-mode .ovf reload: zero-length arena records (empty TEXT/BLOB) were treated as end-of-file; later rows/blocks were never loaded after reopen. Fixed in Storage.ReadAllRecords + default IStorage.ReadAllRecords.
  2. Single-file (.scdb) fixed-width overflow arena: freed blocks serialized as zero-filled gaps misalign the sequential loader; dead slots are now persisted as negative-length tombstone markers tracked across sessions (_deadSlots).
  3. Legacy (1.x-upgrade) variable-length DELETE: only the newest version was tombstoned, so an older stale UPDATE version won the reopen index rebuild and resurrected the deleted row. DELETE now purges every remaining record of the deleted key (single buffered plaintext scan, storage fallback for encrypted files).

Tests

  • New ReopenRoundTripMatrixTests (4 storage variants x insert/update/delete x 3 reopens) + empty-value reopen regressions + legacy delete-after-update regression.
  • Local: SharpCoreDB.Tests 1674/0 (CI filter) + full 1777/0, CQRS 64/0, VectorSearch 143/0, EF Core 116/0, Functional.Linq2DB 24/0.
  • CHANGELOG [2.0.0.2] Fixed updated.

- directory .ovf reload: zero-length arena records are valid; keep scanning so rows after empty TEXT/BLOB values are not silently dropped on reopen (found via SharpCoreDB.CQRS outbox integration tests)

- single-file (.scdb) fixed-width overflow arena: freed slots persist as negative-length tombstone markers tracked across sessions; zero-filled dead gaps misaligned the sequential loader and dropped later values on reopen

- legacy variable-length columnar DELETE: purge every remaining record of a deleted key at delete time so older stale UPDATE versions cannot win the reopen index rebuild and resurrect the row (1.x upgrade path)

- tests: empty-value reopen regressions, 4-variant reopen round-trip matrix, legacy delete-after-update regression; docs: CHANGELOG [2.0.0.2] Fixed
@MPCoreDeveloper
MPCoreDeveloper merged commit 9291cca into master Sep 4, 2026
9 checks passed
@sonarqubecloud

sonarqubecloud Bot commented Sep 4, 2026

Copy link
Copy Markdown

@MPCoreDeveloper
MPCoreDeveloper deleted the fix/reopen-roundtrip-data-integrity branch September 4, 2026 18:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant