Add content-idempotent path upload coordinator - #413
Draft
Hossain007 wants to merge 2 commits into
Draft
Conversation
Adds `IfMatch` and `IfNoneMatch` to the list of allowed arguments for S3 uploads. This enables users to perform conditional uploads, such as preventing the overwrite of existing files by using `IfNoneMatch: "*"`. These parameters are not supported by the `CreateMultipartUpload` API call, so they are added to the blocklist for that specific operation to prevent validation errors. For multipart uploads, these parameters are passed to the `CompleteMultipartUpload` operation. Functional tests have been added for both single-part and multipart uploads to verify that the arguments are passed correctly.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add a small synchronous coordinator for path uploads that compares full-object SHA-256 content before submitting an existing classic or CRT managed transfer.
The coordinator:
HeadObjectwith checksum mode enabled;Falsewithout starting a transfer when content matches;IfNoneMatch='*';IfMatchset to the observed ETag;ChecksumSHA256to the transfer;This PR intentionally supports paths only. It does not add a file-object API.
Stacked dependency
This draft is stacked on #371. The first commit is a rebase of that PR's conditional-write patch onto current
develop; it is not a separate implementation of, or claim of authorship over, #371. After #371 merges, this branch should be rebased so the upstream diff contains only the coordinator and its tests.The proposed Boto3 API is tracked in boto/boto3#4823.
Tests
python -m pytest tests/unit tests/functional -q— 758 passedpre-commit run --all-files— passedThe supported Python matrix and live AWS coverage remain CI/reviewer gates.
Development note
AI tools assisted with implementation and test iteration. Human review is pending while this PR remains a draft.