From d8b39763109cb95049c3344b1c7c9b2215ce8fca Mon Sep 17 00:00:00 2001
From: MPCoreDeveloper <posseth-software@outlook.com>
Date: Fri, 4 Sep 2026 07:03:46 +0200
Subject: [PATCH] docs: session performance summary (tombstones + whole-file
 DML + marker range-read)

---
 docs/CHANGELOG.md                             | 13 +++++++++++++
 .../EXECUTION_PLAN_UPDATE_DELETE.md           | 19 +++++++++++++++++++
 2 files changed, 32 insertions(+)

diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md
index 2046828d..1a49ee31 100644
--- a/docs/CHANGELOG.md
+++ b/docs/CHANGELOG.md
@@ -55,6 +55,19 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
   DELETE (10K of 100K rows): ~0.54s/18.6K ops/s (flush rewrite)  **~0.16s/~64K ops/s (legacy)** and
   **~0.13s/~78K ops/s (fixed-width)**; comparative-harness DELETE (docs table): SQL ~0.82s/~12K 
   **~0.24s/~41K ops/s**, Direct ~0.63s/~16K  **~0.17s/~58K ops/s**  DELETE is back on par with UPDATE.
+- **Whole-file DML resolution + marker range-read (2026-09-03)** ÔÇö batch DELETE/UPDATE on the
+  comparative docs table no longer pays one pread pair per touched row: the small (Ôëñ32 MB,
+  plaintext, legacy variable-length) `.dat` is read once and every target record is resolved from
+  that snapshot (B1 key-only decode for DELETE, raw slice for the UPDATE fastPatch). Positions with
+  an in-batch buffered overwrite are detected via `IStorage.HasBufferedOverwriteAt` and always fall
+  back to the per-record read, so transaction write-behind semantics stay intact. DELETE tombstone
+  markers resolve their record lengths from a single whole-file read instead of one pread per marker.
+  The canonical-DELETE scanner (which never consumed `WHERE`, so the structured batch path was dead
+  code in the harness) is fixed and covered by `CanonicalBatchDelete_EngagesStructuredPath`.
+  Measured (same machine, Release, median of 3): comparative DELETE SQL ~12K  **~59K ops/s** and
+  Direct ~16K  **~86K ops/s**; UPDATE SQL ~35K  **~44K ops/s**, Direct ~49K  **~63K ops/s**;
+  `--pk` DELETE legacy ~64K  **~68K ops/s**, fixed-width ~78K  **~93K ops/s**. Session plan +
+  attribution in `docs/performance/EXECUTION_PLAN_UPDATE_DELETE.md`.
 - **Dedicated SQL batch-INSERT fast path (WP14)** ÔÇö `ExecuteBatchSQL` INSERTs no longer build a
   per-row `Dictionary<string, object>`; VALUES clauses are parsed directly into column-ordered
   `object[]` rows (`PreparedInsertStatement.ParseValuesToArray`) and inserted via the new
diff --git a/docs/performance/EXECUTION_PLAN_UPDATE_DELETE.md b/docs/performance/EXECUTION_PLAN_UPDATE_DELETE.md
index aef6862d..2aa29651 100644
--- a/docs/performance/EXECUTION_PLAN_UPDATE_DELETE.md
+++ b/docs/performance/EXECUTION_PLAN_UPDATE_DELETE.md
@@ -70,6 +70,25 @@ DELETE 10K op de docs-tabel (comparative, Release) — attributie in de structur
 
 **Ontdekte bug (gefixed):** `TryScanCanonicalDml`'s DELETE-tak consumente nooit de whitespace/het `WHERE`-keyword na de tabelnaam → elke canonieke `DELETE ... WHERE col = literal` viel terug op de regex-path → `DeleteMultipleKeys` + B1/W1 (PR #369/#370) waren **dead code in de benchmark-harness**. Fix + diagnostische teller `Database.CanonicalDeleteStatementsParsed` + regressietest `CanonicalBatchDelete_EngagesStructuredPath` (deze PR).
 
+
+## 8. Eindstatus 2026-09-03 (alles gemerged op master `0a9b9fc1`)
+Gemeten op master (Release, zelfde machine; median van 3 runs voor de comparative, 2 voor `--pk`):
+
+| Operatie | SCDB SQL | SCDB Direct | SQLite | opmerking |
+|---|---:|---:|---:|---|
+| INSERT 100K | ~98-102K | ~103K | ~126-138K | al competitief |
+| UPDATE 10K | ~44K | ~63K | ~228K | voor sessie: SQL ~35K / Direct ~49K |
+| DELETE 10K | ~59K | ~86K | ~294K | **voor sessie: SQL ~12K / Direct ~16K** |
+| `--pk` DELETE legacy / FW | 68K / 93K | — | ~320K | FW was ~78K |
+
+**Merged deze sessie (chronologisch):** #367 tombstones · #368 commit-time tombstones · #369 C4+B3 · #370 B1 · #371 canonieke DELETE-scanner-fix + W1 · #372 whole-file DELETE-resolutie · #373 marker range-read · #374 whole-file UPDATE-resolutie.
+
+### Bewuste vervolgstappen (niet in deze sessie, zie secties 3-4)
+1. **Batch-PK stale/lazy-rebuild** na grote delete-batches — grootste open post; vereist eerst PK-index-refresh-infra (`Table.Index` is een plain property zonder lazy rebuild). Ontwerp nodig; daarna kan de per-rij read in DELETE vervallen.
+2. **Commit-marker schrijfbatching** (writes blijven per-offset; range-read zit er al in via #373).
+3. **Fase B (structureel):** fixed-width in-place engine + PageBased als OLTP-default — de weg naar ~1,2-1,5× van SQLite op UPDATE/DELETE.
+4. **Fase C/D (platform):** AOT/R2R als aparte meet-as, median-of-N in de harness, `dotnet-trace` per Fase-B-stap.
+
 **Gevolg voor de attributie:** de per-rij read blijft nodig zolang de delete-core de auto-`rowid`-PK-waarde per rij moet wissen (gate-onderzoek: docs heeft PK-achtige index >1 geregistreerd). Grootste resterende hefbomen, nu met cijfers onderbouwd:
 1. **PK-onderhoud vervangen door één stale/lazy-rebuild** na een grote delete-batch (i.p.v. per-rij `Index.Delete`) → verwijdert ~30-50% van `core` én maakt de per-rij read overbodig (grootste winst op deze workload).
 2. **Marker-batching over een range-read** (commit-tombstones ~50 ms → enkele ms) als vervolg op C4.