EAI-7650: Set explicit request timeout on SeaweedFS S3 route - #800
Open
nowycondro wants to merge 1 commit into
Open
EAI-7650: Set explicit request timeout on SeaweedFS S3 route#800nowycondro wants to merge 1 commit into
nowycondro wants to merge 1 commit into
Conversation
The seaweed HTTPRoute declared no timeouts, so Envoy applied its 15s default request timeout to the S3 data path. Transfers longer than 15s were reset mid-stream, truncating objects regardless of their size. Co-authored-by: Cursor <cursoragent@cursor.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Updates the SeaweedFS S3 HTTPRoute to explicitly set longer Gateway API timeouts so Envoy doesn’t apply its 15s default request timeout and reset long-running object transfers mid-stream.
Changes:
- Add
timeouts.requestandtimeouts.backendRequestset to30mon theseaweedHTTPRoute rule. - Document the rationale inline (duration-based timeout causing truncated objects on slow links).
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
brownzebra
requested changes
Aug 6, 2026
brownzebra
left a comment
Contributor
There was a problem hiding this comment.
Is 15 sec -> 30 minutes really the amount to change? would 1m not be enough?
Instead of hardcoding in the template, I suggest to parametrize it into values and allow for a gitea override.
Contributor
|
OK, so it is based on the whole transfer... would still be nice to have it templated so it could be tuned in the field. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The
seaweedHTTPRoute declares notimeouts, so Envoy applies its 15 second default request timeout to the SeaweedFS S3 data path. Any transfer lasting longer than 15s is reset mid-stream.The limit is duration, not size, which is why it doesn't show up as a body-size setting anywhere. On a slow client link even a modest object fails.
Evidence
Measured against
app-devusing the same 26 MB object and the same presigned URL, varying only transfer speed:Failures surface as
HTTP/2 stream not closed cleanly: INTERNAL_ERROR. Note the response begins asHTTP 200and the stream resets afterwards, so clients see a truncated object rather than a clean error — a silent corruption risk for anything that doesn't verify length or checksum.Confirmed from the live Envoy config dump that the route's
timeoutfield is unset, while routes that need longer already set one explicitly (aiwb-api-routeruns at 10m/30m, AIM routes at 1800s). This route was simply left at the default.Change
Sets
requestandbackendRequestto30mon the S3 route, matching the existing convention for long-running routes on the same gateway. Theseaweed-adminroute is left untouched.Test plan
helm template sources/seaweedfs-configrenders the expectedtimeoutsblockkubectl apply --dry-run=serveraccepts the manifest, validating the30mduration format against the Gateway API schemaContext
This unblocks adapter artifact uploads in AIWB (EAI-7650), where users upload LoRA adapter files directly to storage through a presigned URL. Without this, those uploads fail for anyone whose connection can't finish inside 15 seconds.
Made with Cursor