Skip to content

[ot] Reject out-of-range GSUB/GPOS lookup types instead of truncating - #484

Merged
behdad merged 1 commit into
harfbuzz:mainfrom
yuxi-liu-wired:fix/reject-out-of-range-lookup-type
Oct 4, 2026
Merged

behdad merged 1 commit into
harfbuzz:mainfrom
yuxi-liu-wired:fix/reject-out-of-range-lookup-type

Conversation

@yuxi-liu-wired

@yuxi-liu-wired yuxi-liu-wired commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor

Problem

LookupInfo::new passes the 16-bit lookupType to SubtableInfo::new as lookup_type as u8,
and the extension path does the same with extension_lookup_type() as u8. A malformed font with
lookupType 3586 (0x0E02) is dispatched as type 2 (MultipleSubst), and 7432 (0x1D08) as type 8
(ReverseChainSingleSubst). Those lookups then run over whatever bytes are at the subtable offsets.

HarfBuzz's sanitizer rejects such a GSUB table. HarfRust applies the aliased lookups instead:

input HarfBuzz HarfRust main this PR
"A" 1 glyph, 2 µs 2048 glyphs, 8.5 ms 1 glyph
"Hello, Wörld! fi fl ffi" 23 glyphs, 0.2 ms 38,960 glyphs, 66-290 ms 23 glyphs

Only MAX_LEN_MIN bounds the output, so a short string costs ~10 ms to 1 s per shape() call
for anything that shapes untrusted fonts. perf: 76% in ReverseChainSingleSubstFormat1::apply.

Change

The lookup type stays u16 all the way to the dispatch match, so its existing
_ => return None arm rejects unknown types. Both as u8 truncations are gone.

Tests

Two fuzzer-generated fonts, added to HarfBuzz in harfbuzz/harfbuzz#6319 (in-house
context-matching.tests). Their tests here, context_matching_009..012 in in_house.rs,
are generated from that HarfBuzz branch with gen-shaping-tests.py. All four fail on main and
pass with this change. The tests (default and icu) pass, as do cargo fmt --check and the
1.85, libm and thumbv7em builds. A current clippy reports the same 8 errors on main as on
this branch, all in files this PR doesn't touch.

Found with libFuzzer, then perf and a HarfBuzz trace. This PR was produced by AI agents
(Claude), from the fuzzing and the fix through to this description.

@behdad

behdad commented Oct 2, 2026

Copy link
Copy Markdown
Member

Thanks for this. Would it be possible to add the tests to HarfBuzz and regenerate the HarfRust tests from that, instead of adding them to custom?

behdad pushed a commit to harfbuzz/harfbuzz that referenced this pull request Oct 4, 2026
Two fuzzed fonts whose GSUB has lookups with types like 3586 (0x0E02)
and 7432 (0x1D08). HarfBuzz's sanitizer rejects the table, so the text
shapes to .notdef glyphs with no lookups applied. HarfRust truncated the
type to u8 and ran them as types 2 and 8 (MultipleSubst and
ReverseChainSingleSubst): a single "A" became 2048 glyphs
(harfbuzz/harfrust#484).

Tested with `meson test -C build` (harfbuzz suites: 252 ok, 8 skipped).

Assisted-by: Claude
@yuxi-liu-wired
yuxi-liu-wired force-pushed the fix/reject-out-of-range-lookup-type branch from f7ddb59 to 83c1651 Compare October 4, 2026 20:20
@yuxi-liu-wired

Copy link
Copy Markdown
Contributor Author

Done: the two fonts and their tests are now in HarfBuzz (harfbuzz/harfbuzz#6319, test/shape/data/in-house/tests/context-matching.tests). This branch takes them from there:

  • the fonts move to tests/fonts/in-house/ under their SHA-1 names;
  • context_matching_009 to 012 in in_house.rs come from gen-shaping-tests.py run against that HarfBuzz branch;
  • the custom fuzzer.tests entries and their fonts are removed.

I kept only the new tests in in_house.rs. Regenerating against current HarfBuzz main also adds variations_007–012 and renumbers some vertical_* tests; that's unrelated, so I left it for a separate regeneration.

Also rebased on main: the CHANGELOG entry moved to [Unreleased], since 0.14.0 was released after this PR was opened. The new tests fail on main and pass with the fix. fmt, clippy, the 1.85/libm/thumbv7em builds and the tests (default and icu) pass locally.

@yuxi-liu-wired
yuxi-liu-wired force-pushed the fix/reject-out-of-range-lookup-type branch from 83c1651 to 3225f20 Compare October 4, 2026 20:20
`LookupInfo::new` passed the 16-bit `lookupType` to `SubtableInfo::new`
as `lookup_type as u8`, and the extension path did the same with
`extension_lookup_type()`. A malformed font with e.g. lookupType 3586
(0x0E02) was therefore dispatched as type 2 (MultipleSubst) and 7432
(0x1D08) as type 8 (ReverseChainSingleSubst), running those lookups over
whatever bytes sit at the (equally malformed) subtable offsets.

HarfBuzz's sanitizer rejects such a table outright. HarfRust instead
expanded a single 'A' into 2048 glyphs (bounded only by MAX_LEN_MIN), at
~8 ms per call vs ~2 µs in HarfBuzz, and 23 characters into ~39 000
glyphs in 60-900 ms. Anything shaping untrusted fonts could be made to
burn ~10 ms-1 s per shape() call.

Keep the lookup type as u16 all the way to the dispatch `match`; its
existing `_ => return None` arm now rejects unknown types, and the two
`as u8` truncations go away.

The two fuzzer-generated fonts and their tests come from HarfBuzz
(harfbuzz/harfbuzz#6319, in-house context-matching.tests);
context_matching_009 to 012 are generated from there. All four fail
on main and pass with this change.

Found with libFuzzer + perf.
@yuxi-liu-wired
yuxi-liu-wired force-pushed the fix/reject-out-of-range-lookup-type branch from 3225f20 to e29277b Compare October 4, 2026 20:20
@behdad
behdad self-requested a review October 4, 2026 20:21
@behdad
behdad merged commit 0ea68e9 into harfbuzz:main Oct 4, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants