perf(server): default the inverted index to MaxPendingPostings=Recommended (2M) - #111
Merged
Merged
Conversation
…ended (2M)
The server constructed core/invertedindex with a zero-value Options{}, so
MaxPendingPostings was 0 = UNBOUNDED and the in-memory pending-write buffer grew
until the periodic flush drained it — a noisy ~1.1-1.5 GiB peak build RSS at
linux scale.
Wire the production construction (invertedindexInit) to pass the measured-good
bound RecommendedMaxPendingPostings (2,000,000). Measured with iibench on the
linux corpus (94,559 docs / 41.4M postings, 4x avg): peak build RSS ~1.28 GiB ->
~0.66 GiB (-48%) and, importantly, PREDICTABLE (unbounded is noisy) — for a
measured ~+11% build time and more flush/GC churn. A deliberate memory-over-
build-speed default for the possibly-small-RAM deployment target.
Scope: consumer opt-in only. The core/invertedindex library zero-value contract
(MaxPendingPostings 0 = unbounded) is unchanged, so the published module and its
other consumers are unaffected; only the server sets the bound.
Adds a newInvertedIndex constructor seam so a light unit test captures the Options
the production closure passes and asserts the bound (a regression guard against a
silent revert to unbounded). No on-disk change; search/hits unaffected.
Generated with [Claude Code](https://claude.ai/code)
via [Happy](https://happy.engineering)
Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Happy <yesreply@happy.engineering>
codetrek
approved these changes
Jul 10, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
The server built
core/invertedindexwith a zero-valueOptions{}, soMaxPendingPostingswas 0 = unbounded — the in-memory pending-write buffer grew until the periodic flush drained it, giving a noisy ~1.1–1.5 GiB peak build RSS at scale. This wires the production construction to pass the measured-good boundRecommendedMaxPendingPostings(2,000,000).Why (measured)
iibench on the linux corpus (94,559 docs / 41.4M postings, 4× interleaved avg, prod default vs 2M):
Peak build RSS is roughly halved and made predictable (unbounded is race-dependent and noisy), for a ~+11% build cost and more flush/GC churn — a deliberate memory-over-build-speed default for the possibly-small-RAM deployment target.
Scope
Consumer opt-in only. The
core/invertedindexzero-value contract (MaxPendingPostings0 = unbounded) is unchanged, so the publishedcoremodule and its other consumers are unaffected — only the server sets the bound.git difftouches onlyinternal/server/.A
newInvertedIndexconstructor seam lets a light unit test capture theOptionsthe productioninvertedindexInitclosure passes and assert the bound (a genuine red→green guard against a silent revert to unbounded — verified: dropping the bound makes the test fail withMaxPendingPostings = 0). No on-disk change; search/hits unaffected.Verification
Built via the full SDD flow (spec, 3 review rounds to a clean round, then workflow-driven TDD). Local gates green (go1.24.2 + go.work): build / vet / gofmt /
internal/servertests; and the root go-cov gate: zero CRITICAL, total 94.0% (internal/server95.3%).🤖 Generated with Claude Code