Fix TensorRT engine cache shape collisions - #32563
Aziz Muminov (AzizMuminov) wants to merge 2 commits into
Conversation
|
Azure Pipelines: There may be pipelines that require an authorized user to comment /azp run to run. |
There was a problem hiding this comment.
🟢 Approval recommended
The targeted cache-key correction is consistent with the reported failure and has focused regression coverage.
Pull request overview
Prevents incompatible TensorRT engine-cache reuse by incorporating graph input type and shape metadata into cache identifiers.
Changes:
- Hashes serialized input type metadata when generating TensorRT model IDs.
- Tests that identical graphs with different static shapes produce different IDs.
File summaries
| File | Description |
|---|---|
onnxruntime/core/providers/tensorrt/tensorrt_execution_provider_utils.h |
Adds input type and shape metadata to cache hashing. |
onnxruntime/test/providers/tensorrt/tensorrt_basic_test.cc |
Adds static-shape collision regression coverage. |
Review details
- Files reviewed: 2/2 changed files
- Comments generated: 0
- Review effort level: Balanced
💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
|
Hi maintainers, could someone please authorize the pending fork CI runs, especially Linux TensorRT CI and Windows GPU TensorRT CI, and review this fix for #32254 when convenient? The current head is 3034618. CLA is green, but the 32 GitHub Actions workflows are still marked action_required. Local lint and isolated cache-hash checks pass; the full TensorRT build and regression test still need CI validation. Thank you! |
Description
Fixes #32254.
Include graph input shapes in the TensorRT model/cache identifier so otherwise identical graphs with different static input shapes do not reuse an incompatible engine.
The fingerprint uses rank, ordered static dimensions, and a marker for dynamic dimensions. It uses methods exposed by the shared-provider bridge and deliberately ignores type/dimension denotations and symbolic dimension names. No execution kernels, graph partitioning decisions, public APIs, or provider options are changed.
Add an in-memory Add-model regression test covering 16 shapes (including the reported 3x3 versus 4x4 case), repeated-model ID stability, and annotation-only changes. Existing model-path tests remain unchanged.
Existing TensorRT cache IDs change for shaped inputs, so cached engines may be rebuilt once after adopting this change.
Validation
git diff --checkpass for the two changed C++ files.TRTGenerateIdfunction and repository MurmurHash implementation with lightweight graph stand-ins. It reproduced the collision on the original code and passed 16 shape/rank separation cases, repeated-input stability, directory-independent IDs, and byte-loaded versus file-loaded ID separation with the revised code. This is not a provider build or inference test.