perf: default new columnar PK tables to the fixed-width record layout - #359
Merged
Conversation
added 2 commits
September 2, 2026 21:52
B7+: DatabaseConfig.AutoFixedWidthRecords (default true) makes CREATE TABLE on directory-mode Columnar tables with an explicitly declared PRIMARY KEY use the fixed-width record layout (out-of-line overflow) even when FixedWidthRecordLayout is left false. Effect: keyed INSERT/UPDATE/DELETE become in-place; measured on the fair PK workload (AppendOnly): INSERT +46%, UPDATE +23%, DELETE +16% vs the legacy layout. Scope & safety: - Applies only to tables with a user-declared PK (the implicit _rowid fallback keeps the legacy layout) and only to new tables: the per-table flag is persisted in metadata, existing files are never rewritten, and reopen stays authoritative. - PageBased tables and the single-file (.scdb) layout are untouched by this default. - FixedWidthRecordLayout remains the explicit force/auto-migrate switch for existing databases. Bug fixed while validating the blast radius: the column-ordered batch-INSERT fast path (SerializeRowExact(object[]), used by ExecuteBatchSQL) did not dispatch on the fixed-width flag and wrote legacy variable-length records into fixed-width tables, so batch-inserted rows were invisible to scans (INSERT INTO ... SELECT copied 0 rows). Added a column-ordered FixedWidthCodec.SerializeRow overload and routed the array fast path through it. Tests: full suite 1645/1645. New DirectoryFixedWidthDefaultTests cover default-on for declared-PK tables, legacy for no-PK, opt-out, reopen persistence, and explicit FixedWidthRecordLayout. Legacy-format fixtures (FixedWidthMigrationTests, SqlInPlaceUpdateTests variable-width growth, KnownIssues Issue1 plaintext) now opt out or scan the .ovf arena.
…loads Deduplicates the two SerializeRow value loops into FixedWidthCodec.WriteSlot (Sonar quality gate: duplication on new code <= 3%).
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
B7+: new directory-mode Columnar tables with an explicitly declared PRIMARY KEY now default to the fixed-width record layout (out-of-line overflow) via
DatabaseConfig.AutoFixedWidthRecords(defaulttrue), even whenFixedWidthRecordLayoutis left false. Keyed INSERT/UPDATE/DELETE become in-place overwrites.Measured on the fair PK workload (AppendOnly, 1 run): INSERT +46%, UPDATE +23%, DELETE +16% vs the legacy variable-length layout (see #358, which now also opts the legacy arm out of this default so the comparison stays meaningful).
Scope & safety
_rowidfallback) keep the legacy layout; PageBased tables and single-file (.scdb) are untouched.FixedWidthRecordLayoutremains the explicit force + auto-migrate switch.DatabaseConfig { AutoFixedWidthRecords = false }.Bug found & fixed while validating the blast radius
The column-ordered batch-INSERT fast path (
SerializeRowExact(object[]), used byExecuteBatchSQL) did not dispatch on the fixed-width flag and wrote legacy variable-length records into fixed-width tables → batch-inserted rows were invisible to scans (INSERT INTO dst SELECT …copied 0 rows). Fix: added a column-orderedFixedWidthCodec.SerializeRowoverload and routed the array fast path through the fixed-width codec.Tests
DirectoryFixedWidthDefaultTests: default-on for declared-PK tables, legacy for no-PK tables, config opt-out, format persistence across reopen, explicitFixedWidthRecordLayoutwithout PK.FixedWidthMigrationTests,SqlInPlaceUpdateTestsvariable-width growth) now opt out explicitly;KnownIssuesFixTests.Issue1plaintext check scans the.ovfarena too (TEXT is out-of-line on fixed-width tables).