Skip to content

fix(base64): keep the vector fast paths free of wrapping bounds - #4123

Merged
bobzhang merged 1 commit into
mainfrom
hongbo/base64-bounds
Aug 21, 2026
Merged

fix(base64): keep the vector fast paths free of wrapping bounds#4123
bobzhang merged 1 commit into
mainfrom
hongbo/base64-bounds

Conversation

@bobzhang

Copy link
Copy Markdown
Contributor

Hardening only — no behaviour change for any input a real caller can construct, and no API change.

While reviewing the equivalent code in #4122, Codex CLI flagged two defects in encoding/hex's new vector paths and noted that encoding/base64 shares one of them: "the base64 precedent shares this defect; that does not make it safe." It shares the other as well. This fixes both there.

The two defects

Wrapping block bounds. The loops tested index + 16 <= end. For a BytesView sitting at the very end of a large backing buffer, index + 16 wraps negative and the comparison becomes true, so a short view enters the vector loop and reads past its end. All three sites now bound by subtraction:

while end - index >= 16          // was index + 16 <= end
while end - index >= 3           // was index + 3 <= end
while body - index >= 16 && out.length() - written >= 16

Each subtraction is wrap-free from the view invariants: end = start_offset + length <= src.length() <= Int::MAX with index in [start_offset, end], and in the decoder body/index are view-relative with written <= out.length().

Unchecked output size. encode_v128 sized its UTF-16 buffer as char_count * 2, where char_count is itself full_groups * 4 — two wrapping multiplies feeding an allocation whose vector stores are not bounds checked. A wrapped size means writing outside the buffer rather than failing. The encoder now declines such an input:

guard length <= 0x2000_0000 else { return encode_scalar(bytes, padding~) }

The scalar encoder builds the string through StringBuilder without ever materializing that buffer, and its growth is bounds checked, so it fails safely at sizes the fast path refuses.

Neither defect is reachable through a realistic caller — both need an allocation approaching two gigabytes — but the code they guard is unchecked, so the failure mode is memory corruption rather than a panic. The fix is three bounds and one guard.

Cost

None. Native, this branch vs origin/main, interleaved on the same machine:

bench origin/main this branch bench origin/main this branch
1 220.97 ns 220.87 ns 7 172.83 ns 177.90 ns
2 38.15 ns 38.52 ns 8 72.25 ns 73.91 ns
3 2.86 µs 2.85 µs 9 2.10 µs 2.15 µs
4 211.15 ns 212.43 ns 10 259.58 ns 255.20 ns
5 181.23 µs 180.51 µs 11 448.31 µs 446.44 µs
6 16.59 µs 16.02 µs 12 12.39 µs 12.28 µs

Every figure is within run-to-run noise; the loop-bound rewrite is free.

moon test encoding/base64 passes 21/21 on native and wasm, full moon test 7512/7512, and pkg.generated.mbti is unchanged.

Review

Reviewed by Codex CLI at ultra reasoning effort — approved on the first pass, verdict posted below.

One deliberate gap: the size guard needs a 512 MiB input to reach, so no test exercises it. I asked whether it should be restructured to be testable and the reviewer's answer was no — "exercising it requires impractical CI memory, while existing differential tests cover ordinary block/tail behavior and the boundary itself has a straightforward arithmetic proof."

The block loops tested `index + 16 <= end`, which wraps to a true
comparison for a view sitting at the very end of a large backing buffer,
and `encode_v128` sized its UTF-16 buffer as `char_count * 2` without
checking that the product fits in an `Int`. Neither is reachable through
any realistic caller -- both need an allocation close to two gigabytes --
but the vector loads and stores they guard are not bounds checked, so a
wrapped index reads and writes outside the buffer rather than failing.

Both loops now bound themselves by subtraction, and the encoder hands an
input whose buffer size would wrap to the scalar encoder, which builds
the string without ever materializing that buffer.

Found by Codex CLI while reviewing the equivalent code in #4122, which
fixes the same two defects in `encoding/hex`.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Copilot AI lite review requested due to automatic review settings August 21, 2026 06:23
@bobzhang

Copy link
Copy Markdown
Contributor Author

Codex CLI review (ultra reasoning effort)

Approved on the first pass.

Approve — no blocking findings.

  1. The rewritten bounds are exactly equivalent whenever the original additions do not wrap. Encode still consumes 12-byte vector blocks while at least 16 bytes remain, then identical 3-byte groups and a 0–2-byte tail. Decode likewise preserves its vector/scalar split.

  2. Every subtraction is wrap-free. Valid view invariants keep index <= end/body; decode additionally maintains written <= out.length().

  3. 0x2000_0000 is sound. At that limit, the largest allocation is 1,431,655,768 bytes, below Int::MAX. The exact padding-independent ceiling is 805,306,365 (0x2fff_fffd), but I do not require it—the round bound is simpler and safely conservative. The scalar path's potentially wrapped calculation is only a capacity hint; checked StringBuilder growth traps or fails allocation before any unsafe write.

  4. I found no additional wrapping site. The remaining vector offsets and scalar tail offsets follow from the same invariants. Decode's maximum allocation is 1,610,612,733 bytes, so length / 4 * 3 - padding cannot overflow.

  5. I would not restructure solely to test the guard. Exercising it requires impractical CI memory, while existing differential tests cover ordinary block/tail behavior and the boundary itself has a straightforward arithmetic proof.

Signed-off-by: Codex CLI codex@openai.com

I left the round 0x2000_0000 bound rather than the exact 0x2fff_fffd ceiling — the reviewer explicitly did not require the tighter one, and the round number is easier to read against the comment that explains it.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Hardens the encoding/base64 SIMD (v128) fast paths against integer-wrapping loop bounds and potential output-buffer size wrapping, preserving existing behavior while preventing memory-unsafe out-of-bounds reads/writes on extreme inputs.

Changes:

  • Adds an input-length guard in encode_v128 to avoid wrapped output-buffer sizing before unchecked vector stores.
  • Rewrites SIMD loop bounds in encode_v128 to use subtraction (end - index >= ...) instead of potentially-wrapping addition (index + ... <= end).
  • Rewrites the SIMD loop bound in decode_v128 to use wrap-free subtraction for both input and output bounds.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
encoding/base64/encode_v128.mbt Adds a size guard and rewrites vector-loop bounds to prevent wrapping-based OOB in the encoder fast path.
encoding/base64/decode_v128.mbt Rewrites vector-loop bounds to avoid wrapping checks and keep unchecked stores within the output buffer.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@coveralls

Copy link
Copy Markdown
Collaborator

Coverage Report for CI Build 6263

Coverage decreased (-0.004%) to 90.791%

Details

  • Coverage decreased (-0.004%) from the base build.
  • Patch coverage: 1 uncovered change across 1 file (1 of 2 lines covered, 50.0%).
  • No coverage regressions found.

Uncovered Changes

File Changed Covered %
encoding/base64/encode_v128.mbt 1 0 0.0%
Total (2 files) 2 1 50.0%

Coverage Regressions

No coverage regressions found.


Coverage Stats

Coverage Status
Relevant Lines: 18275
Covered Lines: 16592
Line Coverage: 90.79%
Coverage Strength: 306027.29 hits per line

💛 - Coveralls

@bobzhang
bobzhang merged commit 9ccffb3 into main Aug 21, 2026
17 checks passed
@bobzhang
bobzhang deleted the hongbo/base64-bounds branch August 21, 2026 06:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants