perf: patch the snapshot on drift, cache describe by version - #45
Merged
Conversation
vreshch
marked this pull request as ready for review
August 22, 2026 19:43
An incoming push already computes the changed-path diff, then threw the snapshot away and made the next query walk the whole history again. It now patches instead: the diff says what exists, a log of the range says when it changed, and only a force-push or an unattributable change rebuilds. describe() joins the derived pattern - two spawns once per version, then free, so a page of vault cards costs one computation instead of one per card. The perf gate gains the commit-count axis it was missing (a 4000-commit fast-import fixture, 20000 nightly) and prices the patch against a rebuild measured in the same run. The write path is left alone and pinned at 7 spawns with its own budget test - see the PR body for why.
vreshch
force-pushed
the
perf/snapshot-and-describe
branch
from
August 22, 2026 21:15
66d2951 to
38789e4
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #44 (
feature/read-many) - the perf fixture and the store touch the same files, and the property test bulk-reads throughreadMany. Review after #44, merge after #44.Internal only: no contract change, no new verb, no output shape moves.
S-D items 3 and 6 of the simplification plan.
a. Snapshot patch-on-drift
detectDriftalready computed the changed-path diff on every push and then dropped the snapshot, so the next query walked the entire commit history again. It now patches:--name-status --no-renames) is the authority on what exists at the new versionold..new) is the authority on when each path changed - a wider set than the diff, because a note edited and put back has no diff at all yet did ageOne visible side effect, deliberate: rename detection is off, so an external rename now reports the removed path AND the added one. Observers used to hear only the arrival.
b. describe() cached by version
Two spawns once per version, then free - the store-level version of the derived-cache pattern already used for stats. A returned card is a copy, so a caller's edit cannot become the cache.
c. Write path 7 -> 4/5: NOT done, and why
Measured on this machine, 60 writes, per stage of the 7:
cat-file --batch(existing doc)hash-objectread-tree+update-index+write-treecommit-treeupdate-ref(CAS)The tree trio is the prize (~4ms/write if collapsed to one spawn), and each way to collapse it fails the "provably identical" bar:
mktreefrom the parent tree only builds ONE level. A nested path needsls-tree+mktreeper level (worse forwork/deep/note.md, better only for root-level notes) and it must re-emit every sibling entry with its exact mode - a mistake silently drops files.cat-file --batchfor the parent trees, onehash-object -t tree --stdin-paths) gets to 5 spawns but re-implements git's tree binary format, entry sort order and mode preservation.validateBareRepoTreeexists precisely because repos do contain symlinks and odd modes; a bug here corrupts a user's repository.git fast-importwould do blob + commit + ref in one process, but it force-updates the ref (our compare-and-swap lost-update proof would be replaced by fast-import's own non-fast-forward refusal) and writes a packfile per invocation, on a store that already never gc's.~4ms on the write verb, against corrupting a repo, on the item the plan itself ranked last. Not shipping it. Instead the cost is now pinned by a test that asserts the exact 7-spawn sequence (and that a no-op write stops after the first), so any future attempt is visible in the diff rather than hidden in the numbers.
Conformance-first
The kit carries every claim; both stores pass it.
ConformanceTarget.reopen(new, optional): another instance over the same storage. New proof - a store that followed the changes answers exactly like one that just opened: same list at three shapes, same search, same describe, same read.mutateExternallynow cycles add -> rewrite -> delete, so the kit's external proofs see every kind of drift instead of one append.New
test/stores/snapshot-patch.test.ts:refresh(): the followed store and a fresh one must be equal on list x3, search, describe andreadMany. 80 runs locally,FUZZ_DIFF_RUNSin CI.diff,merge-base,log; a rewind showsls-tree(rebuild).Both guards were mutation-checked: dropping the ancestry check or dating from the diff instead of the range log makes them fail (the property test found the restore case on its own).
Perf (harness, same machine, PERF_SCALE=1000, PERF_COMMITS=4000)
The gate gained the commit-count axis the review asked for: a fast-import churn fixture (4000 commits over 200 notes; 20000 nightly). The drift row is priced against a rebuild measured in the same run, so it stays honest on a slow runner.
Verify: type-check + lint + format + 260 tests + coverage + build + dist smoke green locally; nightly-scale run (20000 commits, 25 differential sequences) green too.
No version bump - the coordinator owns the release.