Guard dim_t overflow in compute_size, reshape and grow - #2097
Open
kyo-zzz wants to merge 1 commit into
Open
Conversation
The model loader validates shapes with a per-multiplication overflow check (since OpenNMT#2091), but the StorageView API paths still multiply dims unchecked: compute_size() and StorageView::reshape() can wrap int64_t (e.g. a shape of {2^62+1, 8} wraps to 8), yielding a view whose shape claims far more elements than _size, and grow() can wrap its addition. Apply the same per-multiplication guard used by the loader to compute_size() and reshape(), add an overflow check to grow(), and reject the overflowing cases with std::invalid_argument in two new unit tests, mirroring the OpenNMT#2094 test style.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
compute_size()(include/ctranslate2/storage_view.h) and the inline product inStorageView::reshape()multiply dimensions without overflow checks, andgrow()adds unchecked. The model loader validates shapes with a per-multiplication guard since #2091, but theStorageViewAPI paths still accept shapes whose true product overflowsdim_t.Concretely, a shape of
{2^62 + 1, 8}has a true product of2^65 + 8, which wraps to 8 in int64 arithmetic — verified against currentmain.reserve(8)then succeeds and the resulting view claims2^62 + 1rows while_sizeis 8, a shape/size inconsistent state that downstream ops cannot reason about. The existing #2094 guards only coverreserve()itself, so they cannot catch a size that wrapped before entering it.What changed
compute_size(): reject negative dimensions and apply the same per-multiplication guard the model loader uses (size > max / dim).StorageView::reshape(): same guard for its inlineknown_size *= dimproduct.StorageView::grow(): rejectsize < 0and_shape[dim] + size > maxbefore the addition.tests/storage_view_test.ccmirroring the Check for overflow in StorageView::reserve before allocation #2094 test style: an overflowing shape must throw from theStorageViewconstructor, and an overflowingreshape()target must throw.This is defense-in-depth consistent with the current hardening wave (#2068, #2073, #2091, #2094). I audited the remaining model-parsing surface while preparing this:
consume<is used only insrc/models/model.cc, the variable loop is covered by the #2091 guard, string/rank reads are length-bounded,vocabulary.ccreserves are driven by in-memory vector sizes, and the 32-bitconsume<uint32_t> * item_sizeproduct on the pre-v4 path can only shrinknum_bytes, which the downstream int64 equality check against the true product rejects — so I found no other exploitable site.Testing
BUILD_TESTS=ON,OPENMP_RUNTIME=COMP,BUILD_SHARED_LIBS=OFF.ctranslate2_test. Note: the gtest binary crashes at startup (0xC0000409, even for--gtest_list_tests) in my local environment withENABLE_CPU_DISPATCH=ON; the same crash reproduces on unmodifiedmain, so it is a local toolchain quirk rather than something this patch introduces. The wrapping behavior itself was verified with a standalone program againstcompute_size()on currentmain(returns 8 for the shape above).storage_view_test.cctests are untouched and the diff adds no new failure modes on the happy path (valid shapes are unaffected: the guard only rejects values that would previously wrap).AI usage disclosure
Per the contribution policy: this fix was developed with AI assistance (analysis of the overflow paths, drafting the guards and tests, and building/verifying locally). I have reviewed every line of the diff, understand the wrap arithmetic and the interaction with the #2091/#2094 guards, and take full responsibility for correctness, performance, and design.