Skip to content

feat: LFS signed-url uploads - #32

Open
mvanhorn wants to merge 4 commits into
tobi:mainfrom
mvanhorn:cursor/lfs-signed-url-uploads-3ca3
Open

feat: LFS signed-url uploads#32
mvanhorn wants to merge 4 commits into
tobi:mainfrom
mvanhorn:cursor/lfs-signed-url-uploads-3ca3

Conversation

@mvanhorn

@mvanhorn mvanhorn commented Aug 28, 2026

Copy link
Copy Markdown

Walkthrough

HyperFrames walkthrough of this PR.

Walkthrough

mp4

lfs.serve_via = "signed_url" now applies to the upload half of the batch as well: the upload action becomes a presigned PUT straight at the store, the way GitHub and GitLab hand out LFS uploads. Downloads already did this; uploads always proxied through PUT /info/lfs/objects/{oid}. proxy stays the default, and its size + sha256 gate is untouched.

A signed PUT is only ever handed out bound to the oid — the LFS keyspace is content-addressed and served back to everyone as immutable, so a PUT that accepts any bytes is a write primitive for every oid a client can name.

ObjectStore::signed_put_url(key, ttl, checksum_sha256) -> Result<Option<SignedPut>> therefore answers Ok(None) unless the backend both rejects a body whose sha256 differs and keeps the header carrying that checksum inside the signature:

backend answer
S3 x-amz-checksum-sha256, signed. Mismatching body: BadDigest; dropped header: SignatureDoesNotMatch. The URL is discarded if the header ever falls outside X-Amz-SignedHeaders.
GCS None. x-goog-hash validates only CRC32C/MD5, neither of which walgit knows for an object it has never seen, and x-goog-content-sha256 is UNSIGNED-PAYLOAD on the signed-URL path. Uploads stay proxied.
memory None.

A signing failure logs a WARN and falls back to the proxying href; it never fails a push. lfs.max_object_bytes is only enforceable where the bytes pass through, so an object over the cap is not signed and the proxy href refuses it with 413.

verify stays walgit's: the store guarantees the content, we still confirm the object arrived at the promised size. Since authenticated: true keeps git-lfs from putting walgit's credential on the store's URL — and git-lfs applies that flag to the verify POST too (tq/verify.go) — the verify action carries the credential the client used on the batch, as GitHub does. Dropping it makes git lfs push fail with Authorization error: …/info/lfs/verify, which the test below asserts.

No new config key. signed_get_url downloads now also send authenticated: true, so git-lfs stops asking a credential helper for the bucket's host.

Round trips

Request counts are unchanged: S3 presigning is offline, GCS returns without a call, and signed_put_url is only reached under serve_via = "signed_url". What changes is where the bytes go — walgit no longer PUTs the object at all — so docs/ROUNDTRIPS.md §2 gains a row for the LFS batch and one for the signed upload.

Tests

  • crates/walgit-store/src/s3.rs — presigning needs no bucket, so the guarantee is a unit test: the checksum header is required of the client and listed in X-Amz-SignedHeaders.
  • crates/walgit-store/tests/contract.rs — every backend: signed_put_url is bound to the sha256 or absent; memory is absent.
  • crates/walgit-server/tests/lfs_signed_url.rs — the batch response under a signing store, a store that cannot sign, a denied signer, and the cap; plus a real git lfs push against a mock bucket that checks the signed checksum the way S3 does and hands the object to the store, then verify here.
  • just test (with lfs_signed_url added to the tier), just e2e and cargo test -p walgit-server --test sim pass, lfs_upstream included.

Notes

e2e::lfs_roundtrip_when_available fails on this branch and identically on main at 6d8fa54: git-lfs 3.7.1 ignores -c filter.lfs.* on a clone without a local git lfs install, so the checkout keeps the pointer. Left alone. just clippy is likewise red before and after (the 1.97.1 gate added in 5ccc405); this change adds no new finding except one expect() in contract.rs, which already has 66 of them.

cursoragent and others added 3 commits August 28, 2026 16:56
`lfs.serve_via = "signed_url"` now applies to the upload half of the batch as
well: the `upload` action becomes a presigned PUT straight at the store, the way
GitHub and GitLab hand out LFS uploads. Downloads already did this; uploads always
proxied through `PUT /info/lfs/objects/{oid}`. `proxy` stays the default.

A signed PUT is only ever handed out bound to the oid. The LFS keyspace is
content-addressed and served back to everyone as immutable, so a PUT that accepts
any bytes is a write primitive for every oid a client can name.
`ObjectStore::signed_put_url(key, ttl, checksum_sha256)` therefore answers
`Ok(None)` unless the backend both rejects a body whose sha256 differs and keeps
the header carrying that checksum inside the signature:

* S3 signs `x-amz-checksum-sha256` (mismatch: BadDigest; dropped header:
  SignatureDoesNotMatch), and the URL is discarded if the header ever falls
  outside `X-Amz-SignedHeaders`.
* GCS cannot: `x-goog-hash` validates only CRC32C/MD5, neither of which walgit
  knows for an object it has never seen, and `x-goog-content-sha256` is
  UNSIGNED-PAYLOAD on the signed-URL path. Uploads stay proxied there.
* memory signs nothing.

`verify` stays walgit's either way. Since `authenticated: true` keeps git-lfs
from putting walgit's credential on the store's URL — and git-lfs applies that
flag to the verify POST too — the verify action carries the credential the client
used on the batch. `lfs.max_object_bytes` is only enforceable where the bytes
pass through, so an object over the cap is not signed and the proxy href refuses
it. The proxy path's own size + sha256 gate is untouched.

Co-authored-by: Matt Van Horn <mvanhorn@users.noreply.github.com>
… bucket

The memory store's fake signed PUT takes a base URL instead of a boolean, so a
test can point it at a bucket it controls. `lfs_signed_url.rs` runs a real
`git lfs push` at one that checks the signed checksum the way S3 does, then
asserts the object landed and `verify` came back to walgit.

Both halves of the `authenticated` decision are pinned: with the flag dropped the
shape test fails, and with the credential off the `verify` action git-lfs reports
"Authorization error: .../info/lfs/verify" and the push fails.

Co-authored-by: Matt Van Horn <mvanhorn@users.noreply.github.com>
A table and a paragraph needed the blank line that separates them from what
precedes.

Co-authored-by: Matt Van Horn <mvanhorn@users.noreply.github.com>
@mvanhorn
mvanhorn marked this pull request as ready for review August 28, 2026 17:33
A signed upload changes the cost model, not the request count: signing is local
crypto, and the object's bytes stop passing through walgit entirely.

Co-authored-by: Matt Van Horn <mvanhorn@users.noreply.github.com>
@mvanhorn

mvanhorn commented Aug 28, 2026

Copy link
Copy Markdown
Author

HyperFrames walkthrough of this PR.

Walkthrough

mp4

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants