Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions docs/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,19 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
DELETE (10K of 100K rows): ~0.54s/18.6K ops/s (flush rewrite)  **~0.16s/~64K ops/s (legacy)** and
**~0.13s/~78K ops/s (fixed-width)**; comparative-harness DELETE (docs table): SQL ~0.82s/~12K 
**~0.24s/~41K ops/s**, Direct ~0.63s/~16K  **~0.17s/~58K ops/s**  DELETE is back on par with UPDATE.
- **Whole-file DML resolution + marker range-read (2026-09-03)** ÔÇö batch DELETE/UPDATE on the
comparative docs table no longer pays one pread pair per touched row: the small (Ôëñ32 MB,
plaintext, legacy variable-length) `.dat` is read once and every target record is resolved from
that snapshot (B1 key-only decode for DELETE, raw slice for the UPDATE fastPatch). Positions with
an in-batch buffered overwrite are detected via `IStorage.HasBufferedOverwriteAt` and always fall
back to the per-record read, so transaction write-behind semantics stay intact. DELETE tombstone
markers resolve their record lengths from a single whole-file read instead of one pread per marker.
The canonical-DELETE scanner (which never consumed `WHERE`, so the structured batch path was dead
code in the harness) is fixed and covered by `CanonicalBatchDelete_EngagesStructuredPath`.
Measured (same machine, Release, median of 3): comparative DELETE SQL ~12K  **~59K ops/s** and
Direct ~16K  **~86K ops/s**; UPDATE SQL ~35K  **~44K ops/s**, Direct ~49K  **~63K ops/s**;
`--pk` DELETE legacy ~64K  **~68K ops/s**, fixed-width ~78K  **~93K ops/s**. Session plan +
attribution in `docs/performance/EXECUTION_PLAN_UPDATE_DELETE.md`.
- **Dedicated SQL batch-INSERT fast path (WP14)** ÔÇö `ExecuteBatchSQL` INSERTs no longer build a
per-row `Dictionary<string, object>`; VALUES clauses are parsed directly into column-ordered
`object[]` rows (`PreparedInsertStatement.ParseValuesToArray`) and inserted via the new
Expand Down
19 changes: 19 additions & 0 deletions docs/performance/EXECUTION_PLAN_UPDATE_DELETE.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,25 @@ DELETE 10K op de docs-tabel (comparative, Release) — attributie in de structur

**Ontdekte bug (gefixed):** `TryScanCanonicalDml`'s DELETE-tak consumente nooit de whitespace/het `WHERE`-keyword na de tabelnaam → elke canonieke `DELETE ... WHERE col = literal` viel terug op de regex-path → `DeleteMultipleKeys` + B1/W1 (PR #369/#370) waren **dead code in de benchmark-harness**. Fix + diagnostische teller `Database.CanonicalDeleteStatementsParsed` + regressietest `CanonicalBatchDelete_EngagesStructuredPath` (deze PR).


## 8. Eindstatus 2026-09-03 (alles gemerged op master `0a9b9fc1`)
Gemeten op master (Release, zelfde machine; median van 3 runs voor de comparative, 2 voor `--pk`):

| Operatie | SCDB SQL | SCDB Direct | SQLite | opmerking |
|---|---:|---:|---:|---|
| INSERT 100K | ~98-102K | ~103K | ~126-138K | al competitief |
| UPDATE 10K | ~44K | ~63K | ~228K | voor sessie: SQL ~35K / Direct ~49K |
| DELETE 10K | ~59K | ~86K | ~294K | **voor sessie: SQL ~12K / Direct ~16K** |
| `--pk` DELETE legacy / FW | 68K / 93K | — | ~320K | FW was ~78K |

**Merged deze sessie (chronologisch):** #367 tombstones · #368 commit-time tombstones · #369 C4+B3 · #370 B1 · #371 canonieke DELETE-scanner-fix + W1 · #372 whole-file DELETE-resolutie · #373 marker range-read · #374 whole-file UPDATE-resolutie.

### Bewuste vervolgstappen (niet in deze sessie, zie secties 3-4)
1. **Batch-PK stale/lazy-rebuild** na grote delete-batches — grootste open post; vereist eerst PK-index-refresh-infra (`Table.Index` is een plain property zonder lazy rebuild). Ontwerp nodig; daarna kan de per-rij read in DELETE vervallen.
2. **Commit-marker schrijfbatching** (writes blijven per-offset; range-read zit er al in via #373).
3. **Fase B (structureel):** fixed-width in-place engine + PageBased als OLTP-default — de weg naar ~1,2-1,5× van SQLite op UPDATE/DELETE.
4. **Fase C/D (platform):** AOT/R2R als aparte meet-as, median-of-N in de harness, `dotnet-trace` per Fase-B-stap.

**Gevolg voor de attributie:** de per-rij read blijft nodig zolang de delete-core de auto-`rowid`-PK-waarde per rij moet wissen (gate-onderzoek: docs heeft PK-achtige index >1 geregistreerd). Grootste resterende hefbomen, nu met cijfers onderbouwd:
1. **PK-onderhoud vervangen door één stale/lazy-rebuild** na een grote delete-batch (i.p.v. per-rij `Index.Delete`) → verwijdert ~30-50% van `core` én maakt de per-rij read overbodig (grootste winst op deze workload).
2. **Marker-batching over een range-read** (commit-tombstones ~50 ms → enkele ms) als vervolg op C4.
Expand Down
Loading