feat: LFS signed-url uploads - #32
Open
mvanhorn wants to merge 4 commits into
Open
Conversation
`lfs.serve_via = "signed_url"` now applies to the upload half of the batch as
well: the `upload` action becomes a presigned PUT straight at the store, the way
GitHub and GitLab hand out LFS uploads. Downloads already did this; uploads always
proxied through `PUT /info/lfs/objects/{oid}`. `proxy` stays the default.
A signed PUT is only ever handed out bound to the oid. The LFS keyspace is
content-addressed and served back to everyone as immutable, so a PUT that accepts
any bytes is a write primitive for every oid a client can name.
`ObjectStore::signed_put_url(key, ttl, checksum_sha256)` therefore answers
`Ok(None)` unless the backend both rejects a body whose sha256 differs and keeps
the header carrying that checksum inside the signature:
* S3 signs `x-amz-checksum-sha256` (mismatch: BadDigest; dropped header:
SignatureDoesNotMatch), and the URL is discarded if the header ever falls
outside `X-Amz-SignedHeaders`.
* GCS cannot: `x-goog-hash` validates only CRC32C/MD5, neither of which walgit
knows for an object it has never seen, and `x-goog-content-sha256` is
UNSIGNED-PAYLOAD on the signed-URL path. Uploads stay proxied there.
* memory signs nothing.
`verify` stays walgit's either way. Since `authenticated: true` keeps git-lfs
from putting walgit's credential on the store's URL — and git-lfs applies that
flag to the verify POST too — the verify action carries the credential the client
used on the batch. `lfs.max_object_bytes` is only enforceable where the bytes
pass through, so an object over the cap is not signed and the proxy href refuses
it. The proxy path's own size + sha256 gate is untouched.
Co-authored-by: Matt Van Horn <mvanhorn@users.noreply.github.com>
… bucket The memory store's fake signed PUT takes a base URL instead of a boolean, so a test can point it at a bucket it controls. `lfs_signed_url.rs` runs a real `git lfs push` at one that checks the signed checksum the way S3 does, then asserts the object landed and `verify` came back to walgit. Both halves of the `authenticated` decision are pinned: with the flag dropped the shape test fails, and with the credential off the `verify` action git-lfs reports "Authorization error: .../info/lfs/verify" and the push fails. Co-authored-by: Matt Van Horn <mvanhorn@users.noreply.github.com>
A table and a paragraph needed the blank line that separates them from what precedes. Co-authored-by: Matt Van Horn <mvanhorn@users.noreply.github.com>
mvanhorn
marked this pull request as ready for review
August 28, 2026 17:33
A signed upload changes the cost model, not the request count: signing is local crypto, and the object's bytes stop passing through walgit entirely. Co-authored-by: Matt Van Horn <mvanhorn@users.noreply.github.com>
Author
|
HyperFrames walkthrough of this PR. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Walkthrough
HyperFrames walkthrough of this PR.
mp4
lfs.serve_via = "signed_url"now applies to the upload half of the batch as well: theuploadaction becomes a presigned PUT straight at the store, the way GitHub and GitLab hand out LFS uploads. Downloads already did this; uploads always proxied throughPUT /info/lfs/objects/{oid}.proxystays the default, and its size + sha256 gate is untouched.A signed PUT is only ever handed out bound to the oid — the LFS keyspace is content-addressed and served back to everyone as immutable, so a PUT that accepts any bytes is a write primitive for every oid a client can name.
ObjectStore::signed_put_url(key, ttl, checksum_sha256) -> Result<Option<SignedPut>>therefore answersOk(None)unless the backend both rejects a body whose sha256 differs and keeps the header carrying that checksum inside the signature:x-amz-checksum-sha256, signed. Mismatching body:BadDigest; dropped header:SignatureDoesNotMatch. The URL is discarded if the header ever falls outsideX-Amz-SignedHeaders.None.x-goog-hashvalidates only CRC32C/MD5, neither of which walgit knows for an object it has never seen, andx-goog-content-sha256isUNSIGNED-PAYLOADon the signed-URL path. Uploads stay proxied.None.A signing failure logs a WARN and falls back to the proxying href; it never fails a push.
lfs.max_object_bytesis only enforceable where the bytes pass through, so an object over the cap is not signed and the proxy href refuses it with 413.verifystays walgit's: the store guarantees the content, we still confirm the object arrived at the promised size. Sinceauthenticated: truekeeps git-lfs from putting walgit's credential on the store's URL — and git-lfs applies that flag to the verify POST too (tq/verify.go) — the verify action carries the credential the client used on the batch, as GitHub does. Dropping it makesgit lfs pushfail withAuthorization error: …/info/lfs/verify, which the test below asserts.No new config key.
signed_get_urldownloads now also sendauthenticated: true, so git-lfs stops asking a credential helper for the bucket's host.Round trips
Request counts are unchanged: S3 presigning is offline, GCS returns without a call, and
signed_put_urlis only reached underserve_via = "signed_url". What changes is where the bytes go — walgit no longer PUTs the object at all — sodocs/ROUNDTRIPS.md§2 gains a row for the LFS batch and one for the signed upload.Tests
crates/walgit-store/src/s3.rs— presigning needs no bucket, so the guarantee is a unit test: the checksum header is required of the client and listed inX-Amz-SignedHeaders.crates/walgit-store/tests/contract.rs— every backend:signed_put_urlis bound to the sha256 or absent; memory is absent.crates/walgit-server/tests/lfs_signed_url.rs— the batch response under a signing store, a store that cannot sign, a denied signer, and the cap; plus a realgit lfs pushagainst a mock bucket that checks the signed checksum the way S3 does and hands the object to the store, thenverifyhere.just test(withlfs_signed_urladded to the tier),just e2eandcargo test -p walgit-server --test simpass,lfs_upstreamincluded.Notes
e2e::lfs_roundtrip_when_availablefails on this branch and identically onmainat 6d8fa54: git-lfs 3.7.1 ignores-c filter.lfs.*on a clone without a localgit lfs install, so the checkout keeps the pointer. Left alone.just clippyis likewise red before and after (the 1.97.1 gate added in 5ccc405); this change adds no new finding except oneexpect()incontract.rs, which already has 66 of them.