Skip to content

Guard dim_t overflow in compute_size, reshape and grow - #2097

Open
kyo-zzz wants to merge 1 commit into
OpenNMT:masterfrom
kyo-zzz:hardening/dim-overflow
Open

Guard dim_t overflow in compute_size, reshape and grow#2097
kyo-zzz wants to merge 1 commit into
OpenNMT:masterfrom
kyo-zzz:hardening/dim-overflow

Conversation

@kyo-zzz

@kyo-zzz kyo-zzz commented Sep 2, 2026

Copy link
Copy Markdown

Summary

compute_size() (include/ctranslate2/storage_view.h) and the inline product in StorageView::reshape() multiply dimensions without overflow checks, and grow() adds unchecked. The model loader validates shapes with a per-multiplication guard since #2091, but the StorageView API paths still accept shapes whose true product overflows dim_t.

Concretely, a shape of {2^62 + 1, 8} has a true product of 2^65 + 8, which wraps to 8 in int64 arithmetic — verified against current main. reserve(8) then succeeds and the resulting view claims 2^62 + 1 rows while _size is 8, a shape/size inconsistent state that downstream ops cannot reason about. The existing #2094 guards only cover reserve() itself, so they cannot catch a size that wrapped before entering it.

What changed

  • compute_size(): reject negative dimensions and apply the same per-multiplication guard the model loader uses (size > max / dim).
  • StorageView::reshape(): same guard for its inline known_size *= dim product.
  • StorageView::grow(): reject size < 0 and _shape[dim] + size > max before the addition.
  • Two unit tests in tests/storage_view_test.cc mirroring the Check for overflow in StorageView::reserve before allocation #2094 test style: an overflowing shape must throw from the StorageView constructor, and an overflowing reshape() target must throw.

This is defense-in-depth consistent with the current hardening wave (#2068, #2073, #2091, #2094). I audited the remaining model-parsing surface while preparing this: consume< is used only in src/models/model.cc, the variable loop is covered by the #2091 guard, string/rank reads are length-bounded, vocabulary.cc reserves are driven by in-memory vector sizes, and the 32-bit consume<uint32_t> * item_size product on the pre-v4 path can only shrink num_bytes, which the downstream int64 equality check against the true product rejects — so I found no other exploitable site.

Testing

  • Built with MSVC 19.43 / CUDA 13.0 toolkit, BUILD_TESTS=ON, OPENMP_RUNTIME=COMP, BUILD_SHARED_LIBS=OFF.
  • The two new tests compile into ctranslate2_test. Note: the gtest binary crashes at startup (0xC0000409, even for --gtest_list_tests) in my local environment with ENABLE_CPU_DISPATCH=ON; the same crash reproduces on unmodified main, so it is a local toolchain quirk rather than something this patch introduces. The wrapping behavior itself was verified with a standalone program against compute_size() on current main (returns 8 for the shape above).
  • Existing storage_view_test.cc tests are untouched and the diff adds no new failure modes on the happy path (valid shapes are unaffected: the guard only rejects values that would previously wrap).

AI usage disclosure

Per the contribution policy: this fix was developed with AI assistance (analysis of the overflow paths, drafting the guards and tests, and building/verifying locally). I have reviewed every line of the diff, understand the wrap arithmetic and the interaction with the #2091/#2094 guards, and take full responsibility for correctness, performance, and design.

The model loader validates shapes with a per-multiplication overflow
check (since OpenNMT#2091), but the StorageView API paths still multiply dims
unchecked: compute_size() and StorageView::reshape() can wrap int64_t
(e.g. a shape of {2^62+1, 8} wraps to 8), yielding a view whose shape
claims far more elements than _size, and grow() can wrap its addition.

Apply the same per-multiplication guard used by the loader to
compute_size() and reshape(), add an overflow check to grow(), and
reject the overflowing cases with std::invalid_argument in two new unit
tests, mirroring the OpenNMT#2094 test style.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant