harden read_header against untrusted header_size (CWE-770 unbouded alloc) - #84
Open
li-jin-quan wants to merge 1 commit into
Open
li-jin-quan wants to merge 1 commit into
li-jin-quan wants to merge 1 commit into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
npyz::NpyFile::newparses the.npypre-header and readsheader_sizedirectlyfrom the (attacker-controlled) file. It then allocates
vec![0; header_size]beforereading a single byte of the actual header:
https://github.com/ExpHP/npyz/blob/master/src/header.rs#L222-L231
For
.npyv2.0 / v3.0 theheader_sizefield is au32, so a malicious file can declareup to 4 GiB of header while only supplying a few bytes of body. The parser pre-allocates
the full declared size and only fails later when
read_exactcomes up short — an unboundedallocation driven by an untrusted declared value (CWE-770 / CWE-400, resource
exhaustion / DoS). On platforms where the allocation cannot be satisfied, Rust's
handle_alloc_errorterminates the process, i.e. a hard crash.Impact
Any caller that parses
.npyfiles from an untrusted source (model downloads, useruploads, fuzzing) is exposed to a memory-exhaustion DoS from a tiny input. This is the same
class of bug already fixed in
ndarray-npy(jturner314/ndarray-npy#106).Fix
Grow the buffer from the bytes that are actually present, capped at a small initial
capacity, and only error if the stream is truncated:
Behavior is unchanged for well-formed files (exactly
header_sizebytes are read) and thetruncation path still returns an
io::Error— the only difference is that allocation nowtracks the real data instead of the declared value.
Testing (empirical)
A standalone PoC (
npyzparsing a 13-byte malicious v3.0 file declaring a 256 MiB header)with a counting
#[global_allocator]recorded:The existing test suite is unaffected (the change is a mechanical read-loop equivalent of the
original
vec![0; N] + read_exact).Notes
src/serialize/slice.rs:24(vec![0; self.info.size], wheresizederives from the untrusted dtype/shape) is the same class and could be hardened the same
way in a follow-up; this PR keeps the change minimal and focused on the header pre-header.
Cargo.tomlchange, no magic numbers / hardcoded caps.