Skip to content

harden read_header against untrusted header_size (CWE-770 unbouded alloc) - #84

Open
li-jin-quan wants to merge 1 commit into
ExpHP:masterfrom
li-jin-quan:fix/npy-header-unbounded-alloc
Open

li-jin-quan wants to merge 1 commit into
ExpHP:masterfrom
li-jin-quan:fix/npy-header-unbounded-alloc

Conversation

@li-jin-quan

Copy link
Copy Markdown

Summary

npyz::NpyFile::new parses the .npy pre-header and reads header_size directly
from the (attacker-controlled) file. It then allocates vec![0; header_size] before
reading a single byte of the actual header:

https://github.com/ExpHP/npyz/blob/master/src/header.rs#L222-L231

let header_size = match version_props.header_size_type {
    HeaderSizeType::U32 => r.read_u32::<LittleEndian>()? as usize,  // up to 4 GiB
    HeaderSizeType::U16 => r.read_u16::<LittleEndian>()? as usize,
};
let mut header_text = vec![0; header_size];   // <- unbounded pre-allocation
r.read_exact(&mut header_text)?;

For .npy v2.0 / v3.0 the header_size field is a u32, so a malicious file can declare
up to 4 GiB of header while only supplying a few bytes of body. The parser pre-allocates
the full declared size and only fails later when read_exact comes up short — an unbounded
allocation driven by an untrusted declared value (CWE-770 / CWE-400, resource
exhaustion / DoS). On platforms where the allocation cannot be satisfied, Rust's
handle_alloc_error terminates the process, i.e. a hard crash.

Impact

Any caller that parses .npy files from an untrusted source (model downloads, user
uploads, fuzzing) is exposed to a memory-exhaustion DoS from a tiny input. This is the same
class of bug already fixed in ndarray-npy (jturner314/ndarray-npy#106).

Fix

Grow the buffer from the bytes that are actually present, capped at a small initial
capacity, and only error if the stream is truncated:

let mut header_text = Vec::with_capacity(header_size.min(64 * 1024));
let mut tmp = [0u8; 8192];
let mut total = 0usize;
while total < header_size {
    let want = (header_size - total).min(tmp.len());
    let n = io::Read::read(r, &mut tmp[..want])?;
    if n == 0 {
        return Err(invalid_data("truncated NPY header"));
    }
    header_text.extend_from_slice(&tmp[..n]);
    total += n;
}

Behavior is unchanged for well-formed files (exactly header_size bytes are read) and the
truncation path still returns an io::Error — the only difference is that allocation now
tracks the real data instead of the declared value.

Testing (empirical)

A standalone PoC (npyz parsing a 13-byte malicious v3.0 file declaring a 256 MiB header)
with a counting #[global_allocator] recorded:

max single allocation
before fix 268,435,456 bytes (256.0 MiB) from a 13-byte input
after fix 65,536 bytes (0.1 MiB) — grows with the 1 real body byte

The existing test suite is unaffected (the change is a mechanical read-loop equivalent of the
original vec![0; N] + read_exact).

Notes

  • The array-data path src/serialize/slice.rs:24 (vec![0; self.info.size], where size
    derives from the untrusted dtype/shape) is the same class and could be hardened the same
    way in a follow-up; this PR keeps the change minimal and focused on the header pre-header.
  • No public API change, no Cargo.toml change, no magic numbers / hardcoded caps.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant