fix(docker): persist the TRT-LLM wheel - #4153
Merged
Merged
Conversation
Signed-off-by: Teodor-Dumitru Ene <teodord.ene@gmail.com>
|
Auto-sync is disabled for ready for review pull requests in this repository. Workflows must be run manually. Contributors can view more details about this message here. |
tdene
added
CI:Lfast
Contributor
Author
|
/ok to test c2447db |
kajalj22
self-requested a review
September 16, 2026 15:35
kajalj22
approved these changes
Sep 16, 2026
Contributor
Author
|
/ok to test 84dae7b |
terrykong
approved these changes
Sep 17, 2026
yfw
added a commit
that referenced
this pull request
Sep 17, 2026
…d by #4153 main's #4153 makes the release stage read the mirrored wheel from /opt/trtllm_wheels directly, so the re-seed block this branch added (168aaac) is redundant there -- and after the merge it referenced TRTLLM_WHEEL_CACHE_MIRROR_DIR, which that stage no longer exports, which would have tripped `set -u`. The hermetic stage still writes the mirror. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Signed-off-by: Yi-Fu Wu <yifu.wu@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do ?
The hermetic TRT-LLM step writes the compiled wheel into a cache mount on the builder node. A later image layer build step copies the compiled wheel back into the image.
This causes the TRT-LLM venv build to fail if it runs anywhere other than the original builder node, because the builder will fetch cached layers of the image, then try to finish the build, but it cannot copy the compiled wheel because it's now on an empty mount. This type of layer cache hit happens whenever
uv.lockis not updated, allowing previous images' layers to be reused.#2420 originally introduced this issue, but #4002 exposed it. Before #4002, a failed venv could cause an image build that had a broken venv; after #4002, any failed venv will fail the entire image build process.
This change copies the TRT-LLM wheel into the image on the same layer as it is created, preventing this type of failure. This does not increase the image size because the wheel was already being copied in, just on a later layer. All this does is make sure that a layer cache hit will not cause a failure.
Issues
List issues that this PR closes (syntax):
Usage
# Add a code snippet demonstrating how to use thisBefore your PR is "Ready for review"
Pre checks:
Additional Information