perf: key-only DELETE row decode (B1)

docs/CHANGELOG.md

@@ -42,17 +42,19 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
   overflow arena is reclaimed on the next explicit VACUUM/compaction. Regression: delete half the
   rows, `Flush`, reopen ÔÇö exactly the remaining rows come back. Measured DELETE in the `--pk`
   harness now includes this durability rewrite (~18.6K ops/s when deleting 10K of 100K rows).
-- **Durable DELETE is now in-place via tombstones (no rewrite)** ÔÇö non-transactional Columnar
-  deletes write a tombstone marker (the record's 4-byte length prefix is replaced by the NEGATIVE
-  slot size) instead of queueing a flush-time rewrite, and every raw record enumerator/compactor
-  skips the slot, so DELETE survives a reopen in O(delete). Flush/dispose compaction now only
-  covers transactional deletes (a marker written before commit would survive a rollback that
-  restores the row in the PK index; those deletes are compacted against the current PK after commit
-  ÔÇö rollback-safe). Tombstoned space is reclaimed by the tombstone-aware `CompactTable`, including
-  the ULID-migration compaction (which previously produced an empty file when tombstones were
-  present because `CompactTable` broke on a negative prefix). Measured `--pk` DELETE (10K of 100K
-  rows): ~0.54s/18.6K ops/s (flush rewrite)  **~0.16s/~64K ops/s (legacy)** and
-  **~0.12s/~81K ops/s (fixed-width)** ÔÇö DELETE is back on par with UPDATE.
+- **Durable DELETE is now in-place via tombstones (no rewrite)** ÔÇö Columnar deletes write a
+  tombstone marker (the record's 4-byte length prefix is replaced by the NEGATIVE slot size)
+  instead of queueing a flush-time rewrite, and every raw record enumerator/compactor skips the
+  slot, so DELETE survives a reopen in O(delete). Non-transactional deletes write the marker at
+  delete time; transactional deletes (e.g. any `ExecuteBatchSQL` batch, which runs inside a storage
+  transaction) buffer the offsets and apply the markers at COMMIT ÔÇö rollback discards the buffer,
+  so a rolled-back delete keeps its row, and the flush-time full-file rewrite (`CompactPendingDeletes`)
+  is no longer on the batch-DELETE path. Tombstoned space is reclaimed by the tombstone-aware
+  `CompactTable`, including the ULID-migration compaction (which previously produced an empty file
+  when tombstones were present because `CompactTable` broke on a negative prefix). Measured `--pk`
+  DELETE (10K of 100K rows): ~0.54s/18.6K ops/s (flush rewrite)  **~0.16s/~64K ops/s (legacy)** and
+  **~0.13s/~78K ops/s (fixed-width)**; comparative-harness DELETE (docs table): SQL ~0.82s/~12K 
+  **~0.24s/~41K ops/s**, Direct ~0.63s/~16K  **~0.17s/~58K ops/s**  DELETE is back on par with UPDATE.
 - **Dedicated SQL batch-INSERT fast path (WP14)** ÔÇö `ExecuteBatchSQL` INSERTs no longer build a
   per-row `Dictionary<string, object>`; VALUES clauses are parsed directly into column-ordered
   `object[]` rows (`PreparedInsertStatement.ParseValuesToArray`) and inserted via the new

src/SharpCoreDB/DataStructures/Table.CRUD.cs

@@ -2655,11 +2655,16 @@ private bool HasExplicitNamedIndex(string column)
 
             if (this.storage is { IsInTransaction: true })
             {
-                // Transactional delete: writing the tombstone now would survive a rollback that
-                // restores the row in the PK index, so defer the physical removal to the post-commit
-                // flush compaction (rollback-safe: that rewrite is driven by the CURRENT PK, which
-                // still contains the row after a rollback).
-                Interlocked.Add(ref _pendingLogicalDeletes, recordsToDelete.Count);
+                // Transactional delete: buffer the physical offsets so the in-place marker is
+                // applied at COMMIT (rollback discards the buffer). Durable in O(delete) — the
+                // flush-time full-file rewrite is no longer needed for transactional deletes.
+                foreach (var position in positions)
+                {
+                    if (position >= 0)
+                    {
+                        this.storage.BufferTombstoneForCommit(DataFile, position);
+                    }
+                }
             }
             else
             {
@@ -3016,6 +3021,235 @@ internal void DeleteMultiple(List<string> whereConditions)
         }
     }
 
+    /// <summary>
+    /// Structured variant of <see cref="DeleteMultiple"/> used by the SQL batch dispatcher for
+    /// canonical single-row DELETE statements (`DELETE FROM t WHERE col = literal`, as detected by
+    /// the canonical-DML scanner). The WHERE column and raw literal are already known, so the
+    /// per-statement `col = literal` string rebuild and the second <see cref="TryParseSimpleWhereClause"/>
+    /// pass are skipped on the hot path. Semantics mirror <see cref="DeleteMultiple"/> exactly
+    /// (PK fast path → hash-index fast path → generic fallback with a lazily rebuilt WHERE string,
+    /// only reached for unusual shapes).
+    /// </summary>
+    /// <param name="conditions">Column name and RAW literal value (quotes as written in SQL).</param>
+    [MethodImpl(MethodImplOptions.AggressiveOptimization)]
+    internal void DeleteMultipleKeys(List<(string Column, string Literal)> conditions)
+    {
+        if (this.isReadOnly) throw new InvalidOperationException(ReadOnlyDeleteError);
+        if (conditions.Count == 0) return;
+
+        this.rwLock.EnterWriteLock();
+        try
+        {
+            var engine = GetOrCreateStorageEngine();
+            EnsureAllRegisteredIndexesLoaded();
+
+            // B9 single-pass contiguous DELETE consumes WHERE strings; attempt it (without touching
+            // anything) only when every condition is a literal on the PK column, like the caller.
+            if (this.PrimaryKeyIndex >= 0 && AllPkLiteralConditions(conditions))
+            {
+                var wheres = new List<string>(conditions.Count);
+                foreach (var (col, literal) in conditions)
+                {
+                    wheres.Add(col + " = " + literal);
+                }
+
+                if (TryBulkDeleteContiguousFixedWidth(wheres))
+                {
+                    return;
+                }
+            }
+
+            // B1: decode only the columns the delete core touches (PK + loaded hash-index columns).
+            int[] deleteKeyColumns = BuildDeleteKeyColumns();
+
+            var recordsToDelete = new List<(long storagePosition, Dictionary<string, object> row)>();
+
+            foreach (var (col, literal) in conditions)
+            {
+                string value = UnquoteSqlLiteral(literal);
+
+                // Issue #7 PK fast path (structured: no WHERE-string re-parse).
+                if (StorageMode != StorageMode.PageBased &&
+                    this.PrimaryKeyIndex >= 0 &&
+                    string.Equals(col, this.Columns[this.PrimaryKeyIndex], StringComparison.OrdinalIgnoreCase))
+                {
+                    var fastSearch = this.Index.Search(value);
+                    if (fastSearch.Found)
+                    {
+                        var fastData = engine.Read(Name, fastSearch.Value);
+                        if (fastData != null)
+                        {
+                            var fastRow = DeserializeDeleteKeyRow(fastData, deleteKeyColumns) ?? DeserializeRowFromSpan(fastData);
+                            if (fastRow != null)
+                            {
+                                recordsToDelete.Add((fastSearch.Value, fastRow));
+                            }
+                        }
+
+                        continue;
+                    }
+                }
+                // Hash index fast path.
+                if (this.registeredIndexes.ContainsKey(col))
+                {
+                    EnsureIndexLoaded(col);
+                    if (this.hashIndexes.TryGetValue(col, out var hashIndex))
+                    {
+                        var colIdx = this.Columns.IndexOf(col);
+                        if (colIdx >= 0)
+                        {
+                            var key = ParseValueForHashLookup(value, this.ColumnTypes[colIdx]);
+                            if (key != null)
+                            {
+                                foreach (var pos in hashIndex.LookupPositionsUnsafe(key))
+                                {
+                                    var data = engine.Read(Name, pos);
+                                    if (data != null)
+                                    {
+                                        var row = DeserializeDeleteKeyRow(data, deleteKeyColumns) ?? DeserializeRowFromSpan(data);
+                                        if (row != null) recordsToDelete.Add((pos, row));
+                                    }
+                                }
+
+                                continue;
+                            }
+                        }
+                    }
+                }
+
+                // Generic fallback (rare for canonical shapes): rebuild the WHERE string exactly as
+                // the caller would have and mirror DeleteMultiple.
+                string where = col + " = " + literal;
+                if (this.PrimaryKeyIndex >= 0)
+                {
+                    var rows = SelectInternal(where, orderBy: null, asc: true, noEncrypt: false);
+                    var pkCol = this.Columns[this.PrimaryKeyIndex];
+                    foreach (var row in rows)
+                    {
+                        if (row.TryGetValue(pkCol, out var pkValue) && pkValue != null)
+                        {
+                            var searchResult = this.Index.Search(pkValue.ToString() ?? string.Empty);
+                            if (searchResult.Found)
+                                recordsToDelete.Add((searchResult.Value, row));
+                        }
+                    }
+                }
+                else
+                {
+                    foreach (var (storageRef, data) in engine.GetAllRecords(Name))
+                    {
+                        var row = DeserializeRowFromSpan(data);
+                        if (row != null && (string.IsNullOrEmpty(where) || EvaluateSimpleWhere(row, where)))
+                            recordsToDelete.Add((storageRef, row));
+                    }
+                }
+            }
+
+            if (recordsToDelete.Count == 0) return;
+
+            DeleteRecordsCore(recordsToDelete);
+        }
+        finally
+        {
+            this.rwLock.ExitWriteLock();
+        }
+    }
+
+    private bool AllPkLiteralConditions(List<(string Column, string Literal)> conditions)
+    {
+        var pkCol = this.Columns[this.PrimaryKeyIndex];
+        foreach (var (col, _) in conditions)
+        {
+            if (!string.Equals(col, pkCol, StringComparison.OrdinalIgnoreCase))
+            {
+                return false;
+            }
+        }
+
+        return true;
+    }
+
+    /// <summary>
+    /// Mirrors the value normalization of <see cref="TryParseSimpleWhereClause"/> (trim whitespace,
+    /// then trim enclosing <c>'</c>/<c>"</c> quotes) on a raw SQL literal.
+    /// </summary>
+    private static string UnquoteSqlLiteral(string raw)
+    {
+        return raw.AsSpan().Trim().Trim("'\"".AsSpan()).ToString();
+    }
+
+    /// <summary>
+    /// B1: column indexes a delete target actually needs — the PK value (B-tree cleanup) plus every
+    /// loaded hash-index column (key-only hash cleanup). Ascending, de-duplicated.
+    /// </summary>
+    private int[] BuildDeleteKeyColumns()
+    {
+        var list = new List<int>(4);
+        if (this.PrimaryKeyIndex >= 0)
+        {
+            list.Add(this.PrimaryKeyIndex);
+        }
+
+        foreach (var kvp in this.hashIndexes)
+        {
+            int colIdx = this.Columns.IndexOf(kvp.Key);
+            if (colIdx >= 0 && !list.Contains(colIdx))
+            {
+                list.Add(colIdx);
+            }
+        }
+
+        list.Sort();
+        return list.ToArray();
+    }
+
+    /// <summary>
+    /// B1: decodes only the columns listed in <paramref name="wanted"/> (ascending) from a legacy
+    /// variable-length serialized row, skipping the unneeded columns' payload parsing entirely.
+    /// Returns a minimal dictionary (pk + hash-index columns only) so <see cref="DeleteRecordsCore"/>
+    /// performs the identical PK/hash lookups without materializing the full row. Returns null for
+    /// fixed-width layouts (those go through the fixed-width codec) or corrupt rows — callers fall
+    /// back to full-row deserialization in that case.
+    /// </summary>
+    private Dictionary<string, object>? DeserializeDeleteKeyRow(byte[] data, int[] wanted)
+    {
+        if (_fixedWidthRecords || data == null || data.Length == 0 || wanted.Length == 0)
+        {
+            return null;
+        }
+
+        ReadOnlySpan<byte> span = data.AsSpan();
+        int offset = 0;
+        Dictionary<string, object>? row = null;
+        int wi = 0;
+
+        for (int i = 0; i < Columns.Count; i++)
+        {
+            if (offset >= span.Length)
+            {
+                return null;
+            }
+
+            int size = ReadColumnEncodedSize(span, offset, ColumnTypes[i]);
+            if (size <= 0 || offset + size > span.Length)
+            {
+                return null; // corrupt / unexpected layout → full-row fallback
+            }
+
+            if (wi < wanted.Length && wanted[wi] == i)
+            {
+                var value = ReadTypedValueFromSpan(span.Slice(offset), ColumnTypes[i], out _);
+                row ??= new Dictionary<string, object>(wanted.Length);
+                row[Columns[i]] = value;
+                wi++;
+            }
+
+            offset += size;
+        }
+
+        return row;
+    }
+
     /// <summary>
     /// Shared B8/B9 probe: resolves each key's record position through the PK B-tree, requires the
     /// positions to be physically adjacent at the fixed-width stride, reads the whole contiguous
@@ -3214,9 +3448,15 @@ this.storage is null ||
 
         if (this.storage is { IsInTransaction: true })
         {
-            // Transactional delete: defer physical removal to the post-commit flush compaction
-            // (see DeleteRecordsCore — a tombstone would survive a rollback that restores the row).
-            Interlocked.Add(ref _pendingLogicalDeletes, count);
+            // Transactional delete: buffer the physical offsets so the in-place marker is applied
+            // at COMMIT (see DeleteRecordsCore — rollback discards the buffer).
+            foreach (var position in positions)
+            {
+                if (position >= 0)
+                {
+                    this.storage.BufferTombstoneForCommit(DataFile, position);
+                }
+            }
         }
         else
         {

src/SharpCoreDB/DataStructures/Table.Compaction.cs

@@ -8,7 +8,6 @@ namespace SharpCoreDB.DataStructures;
 using SharpCoreDB.Storage.Engines;
 using System;
 using System.Collections.Generic;
-using System.IO;
 using System.Linq;
 using System.Threading.Tasks;
 
@@ -113,7 +112,8 @@ public void CompactPendingDeletes()
     /// present in the data file, making the Columnar logical delete durable across reopen (readers
     /// skip the marker) without rewriting the remaining rows. Rows appended inside an uncommitted
     /// transaction (positions beyond the current file length) are skipped — their delete commits
-    /// with the rest of the transaction.
+    /// with the rest of the transaction. The storage layer batches the in-place marker writes and
+    /// de-duplicates page-cache evictions per page.
     /// </summary>
     private void TombstoneDeletedPositions(long[] positions)
     {
@@ -122,14 +122,7 @@ private void TombstoneDeletedPositions(long[] positions)
             return;
         }
 
-        long fileLength = File.Exists(DataFile) ? new FileInfo(DataFile).Length : 0;
-        foreach (var position in positions)
-        {
-            if (position >= 0 && position + 4 <= fileLength)
-            {
-                this.storage.TombstoneRecord(DataFile, position);
-            }
-        }
+        this.storage.TombstoneRecords(DataFile, positions);
     }
 
     /// <summary>