Skip to content

fix: parallelize LFS batch presence checks - #20

Open
immanuwell wants to merge 1 commit into
tobi:mainfrom
immanuwell:fix/parallel-lfs-batch-heads
Open

fix: parallelize LFS batch presence checks#20
immanuwell wants to merge 1 commit into
tobi:mainfrom
immanuwell:fix/parallel-lfs-batch-heads

Conversation

@immanuwell

@immanuwell immanuwell commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

LFS batch requests checked each object with one store HEAD at a time.
git-lfs sends 100 objects by default, so 20ms store latency made the batch take 2.12s

This runs 16 checks at once and keeps response order the same.
Request count stays flat, sequential depth drops from 100 to 7

Repro:
cargo test -p walgit-server local_presence_checks_are_parallel_and_preserve_batch_order

The test fails on main and passes here in about 0.19s.
It checks present objects, missing objects, and response order too

@0bserver07

Copy link
Copy Markdown

Pulled this and ran it here (macOS, rust 1.97.1, in-memory store): the lfs_upstream suite passes and the new local_presence_checks_are_parallel_and_preserve_batch_order passed five runs in a row at about 0.45 s. The 1 s bound has plenty of room over the ~140 ms the seven rounds actually need, so it shouldn't flake on a slow runner. The ROUNDTRIPS.md row matches the code: same number of HEADs, depth ceil(n/16). Nice one.

@immanuwell

Copy link
Copy Markdown
Contributor Author

the failing warnings + test check is pre-existing on main, not introduced here. The strict clippy gate from 5ccc405 doesn't pass on the current tree - main has been red on this job ever since (5ccc405, #15, #14)

the errors are clippy::pedantic lints in walgit-proto (generated walgit.v1.rs + time/frame modules), untouched by this change - it only parallelizes LFS batch presence checks. e2e passes.

happy to send a separate PR fixing the gate if useful

@ZackKanter

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants