Skip to content

Fix TensorRT engine cache shape collisions - #32563

Open
Aziz Muminov (AzizMuminov) wants to merge 2 commits into
microsoft:mainfrom
AzizMuminov:codex/trt-engine-cache-shape-key
Open

Aziz Muminov (AzizMuminov) wants to merge 2 commits into
microsoft:mainfrom
AzizMuminov:codex/trt-engine-cache-shape-key

Conversation

@AzizMuminov

@AzizMuminov Aziz Muminov (AzizMuminov) commented Sep 11, 2026

Copy link
Copy Markdown

Description

Fixes #32254.

Include graph input shapes in the TensorRT model/cache identifier so otherwise identical graphs with different static input shapes do not reuse an incompatible engine.

The fingerprint uses rank, ordered static dimensions, and a marker for dynamic dimensions. It uses methods exposed by the shared-provider bridge and deliberately ignores type/dimension denotations and symbolic dimension names. No execution kernels, graph partitioning decisions, public APIs, or provider options are changed.

Add an in-memory Add-model regression test covering 16 shapes (including the reported 3x3 versus 4x4 case), repeated-model ID stability, and annotation-only changes. Existing model-path tests remain unchanged.

Existing TensorRT cache IDs change for shaped inputs, so cached engines may be rebuilt once after adopting this change.

Validation

  • Lint and git diff --check pass for the two changed C++ files.
  • A local isolated C++ harness compiled the actual TRTGenerateId function and repository MurmurHash implementation with lightweight graph stand-ins. It reproduced the collision on the original code and passed 16 shape/rank separation cases, repeated-input stability, directory-independent IDs, and byte-loaded versus file-loaded ID separation with the revised code. This is not a provider build or inference test.
  • The TensorRT provider and new regression test have not been built/run locally: this macOS host has no CUDA/TensorRT toolchain. TensorRT CI validation is still required.

Copilot AI balanced review requested due to automatic review settings September 11, 2026 16:12
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The targeted cache-key correction is consistent with the reported failure and has focused regression coverage.

Pull request overview

Prevents incompatible TensorRT engine-cache reuse by incorporating graph input type and shape metadata into cache identifiers.

Changes:

  • Hashes serialized input type metadata when generating TensorRT model IDs.
  • Tests that identical graphs with different static shapes produce different IDs.
File summaries
File Description
onnxruntime/core/providers/tensorrt/tensorrt_execution_provider_utils.h Adds input type and shape metadata to cache hashing.
onnxruntime/test/providers/tensorrt/tensorrt_basic_test.cc Adds static-shape collision regression coverage.
Review details
  • Files reviewed: 2/2 changed files
  • Comments generated: 0
  • Review effort level: Balanced

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@AzizMuminov

Copy link
Copy Markdown
Author

Hi maintainers, could someone please authorize the pending fork CI runs, especially Linux TensorRT CI and Windows GPU TensorRT CI, and review this fix for #32254 when convenient? The current head is 3034618. CLA is green, but the 32 GitHub Actions workflows are still marked action_required. Local lint and isolated cache-hash checks pass; the full TensorRT build and regression test still need CI validation. Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

TensorRT EP engine cache reuses engine for same graph name with different static input shape

2 participants