feat(text): per-character font fallback - #387
Conversation
…x/abspos content - PAGINATION-FORCED-OVERFLOW-001 fired on every page of any document whose content sits in one wrapper taller than a page: the wrapper is committed at the page top only to be entered, and its children paginate normally. The top-level report is now deferred for an enterable block-flow / flex wrapper and emitted only if its content really did not paginate (using the emitted extent on a resumed page). A nested box whose OWN border box is taller than a page is reported where it is placed, once. - Chasing the remaining reports found real content loss: a `flex: 1` list in a stretched column card was flexed to ~0, laid out into that budget, paginated, and everything after the first item was discarded (02-travel-quote). Column items with auto height/min-height now honor the 4.5 automatic minimum (grown to their content height, shared with the pre-measure), and item content measured against the item's own size never paginates (NestedContentMeasurer suppressPagination). - Absolutely positioned content is laid out with pagination suppressed, like fixed content, so content taller than the box overflows it instead of being cut after the first break. Corpus: 02-travel-quote now shows every card feature (was 1 of 5-6); no other page changes. Forced-overflow reports drop from ~160 to 1 across the 28 docs (the remaining one is the measure-width over-estimate, next PR). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The block measure pass (MeasureSubtreeVisualBlockExtentRecursive) laid every nested text block, flex row, table and multicol out at the BFC (page) content width. Text that wraps inside a narrow box was measured as one line, so: - a narrow box's painted border was too short (lost its bottom padding); - spacing after such content was squeezed (index.pdf payment block: 4px gap instead of the browser's 42px); - the mid-split entry under-measured a list's first item, entered the list at the page bottom, and line-split it leaving a single line. The containing content width is now threaded through the recursion (MeasureInlineGeometry mirrors the emit path's ResolveInFlowBorderBoxInlineSize: explicit / % width, box-sizing, min/max, else fill minus margins), and used by the inline-only, flex, table-wrapper and multicol branches, the first-child estimate, and the keep-with-next lookahead. Also makes HtmlPdfFacadeTests' slow-loader timeout test deterministic: it waits for the deadline to pass instead of a fixed 600ms delay (a busy test host could fire the 100ms timer late and the render finished first). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…c/.otc) The system-font indexer skipped collection files, so families that ship only as a collection (macOS Helvetica, Helvetica Neue, Avenir, Menlo, Optima; Windows/Linux CJK families) fell back to another, often wider, font. - FontCollection: bounded random-access reader for the ttcf header and per-face directories, and ExtractFace, which copies one face into a standalone sfnt (tag-sorted directory, 4-byte aligned tables, recomputed head.checkSumAdjustment) so the validator, parser, HarfBuzz and the PDF subsetter need no collection awareness. - The enumerator detects collections by content and indexes each face from its name/OS/2/head tables only, skipping faces with rejected tables, missing required tables, or over the byte cap. - SystemFontResolver extracts and validates the chosen face (one cache slot per face); a face whose full parse fails resolves to nothing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each text run used one font, so characters it lacked (✔, ▲, →, other scripts) were drawn as .notdef boxes even when a later font-family entry or an installed symbol font had them. - LineBuilder.Shape: a run that shaped to visible .notdef is split by font coverage; each piece is shaped with the first font of the style's fallback chain that covers it (ItemizedRun.FontIndex). Marks, variation selectors and format characters follow their neighbour's font. - IShaperResolver gains Resolve(style, fontIndex) and FindFallbackFont (default: no fallback). HarfBuzzShaperResolver builds the chain once per query: rest of the author stack, default family, then a fixed list of symbol / broad-coverage system families; each validated, de-duped by content, unusable candidates skipped. - TextPainter embeds glyphs from the run's own font and keeps the primary font's metrics for the baseline. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d83eaf9ace
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if (FollowsNeighbourFont(cp)) font = -1; | ||
| else if (primary.HasGlyph(cp) || char.IsWhiteSpace(concatText[i]) || char.IsControl(concatText[i])) font = 0; | ||
| else font = resolver.FindFallbackFont(style, cp); | ||
| if (font > 0) anyFallback = true; | ||
| for (var k = 0; k < width; k++) fontOf[i - start + k] = font; | ||
| i += width; | ||
| } | ||
| return output; | ||
| if (!anyFallback) return null; |
There was a problem hiding this comment.
When the primary font covers a base character but lacks its combining mark, the mark is assigned -1 before any coverage lookup and is later forced back onto the base's primary font. Even if a later font covers both codepoints, anyFallback remains false and the original run is kept, so strings such as a Latin base plus an uncommon combining mark still render tofu. Resolve coverage for the whole grapheme cluster and select a font that can render all of it rather than unconditionally inheriting the base's initial font.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Fixed in 8f284a1. Fallback is now chosen per extended grapheme cluster (StringInfo, UAX #29): a cluster the primary doesn't fully cover goes to the first fallback that covers ALL of it; only when none does, it stays with the font that covers its base. Test: A_cluster_goes_to_the_font_that_covers_all_of_it (primary has A but not U+0301; a later font has both → the whole cluster moves).
| private static bool FollowsNeighbourFont(int codepoint) | ||
| { | ||
| if (codepoint is >= 0xFE00 and <= 0xFE0F or >= 0xE0100 and <= 0xE01EF) return true; // variation selectors | ||
| var category = CharUnicodeInfo.GetUnicodeCategory(codepoint); | ||
| return category is UnicodeCategory.NonSpacingMark or UnicodeCategory.SpacingCombiningMark |
There was a problem hiding this comment.
If a run contains a lone surrogate that shapes to .notdef, the new fallback scan passes that surrogate value to CharUnicodeInfo.GetUnicodeCategory, which requires a valid Unicode scalar and throws instead of preserving HbShaper's documented replacement-character behavior. This can make conversion fail for malformed input precisely when the primary font lacks a replacement glyph; treat unpaired surrogates as ordinary unsupported characters or normalize them to U+FFFD before category lookup.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Fixed in 8f284a1. Codepoints are decoded with Rune.DecodeFromUtf16; a malformed cluster (lone surrogate) stays on the primary font, so HarfBuzz's replacement behavior is unchanged and nothing throws. Test: A_lone_surrogate_stays_on_the_primary_instead_of_throwing.
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
Cluster-level fallback, unusable-font handling, and eager system-font loading have unresolved correctness and performance issues.
Review effort: Balanced
Findings: 1
Open (3)
What changed in this PR
Adds per-character font fallback across shaping, layout, and PDF embedding.
Changes:
- Splits missing-glyph runs across fallback fonts.
- Builds and caches author/system fallback chains.
- Adds fallback coverage, embedding, diagnostics, and tests.
| File | Description |
|---|---|
src/NetPdf.Layout/Inline/IShaperResolver.cs |
Extends fallback resolution API. |
src/NetPdf.Layout/Inline/ItemizedRun.cs |
Tracks each run’s font index. |
src/NetPdf.Layout/Inline/LineBuilder.cs |
Splits and shapes text by coverage. |
src/NetPdf.Text/Shaping/HbShaper.cs |
Adds nominal-glyph coverage checks. |
src/NetPdf/Shaping/HarfBuzzShaperResolver.cs |
Builds and caches fallback chains. |
src/NetPdf/Rendering/TextPainter.cs |
Embeds fallback glyphs with primary metrics. |
tests/NetPdf.UnitTests/Shaping/FontFallbackTests.cs |
Tests fallback behavior end-to-end. |
tests/NetPdf.UnitTests/Text/Fonts/OpenType/SyntheticFont.cs |
Supports configurable synthetic coverage. |
tests/NetPdf.UnitTests/Text/Fonts/Woff/SyntheticWoff.cs |
Corrects XML reference syntax. |
tests/NetPdf.UnitTests/Text/Fonts/FontMetadataWeightClampTests.cs |
Corrects XML reference syntax. |
docs/compatibility-matrix.md |
Documents per-character fallback. |
CHANGELOG.md |
Announces the feature. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| if (face is null || face.Bytes.IsEmpty) return null; | ||
| var verdict = FontSafetyValidator.Validate(face.Bytes.Span); | ||
| if (!verdict.IsSafe | ||
| || verdict.DetectedFormat is FontSafetyValidator.FontFormat.Woff or FontSafetyValidator.FontFormat.Woff2) | ||
| return null; | ||
| return new ResolvedFontProgram(Convert.ToHexString(SHA256.HashData(face.Bytes.Span)), face.Bytes); |
There was a problem hiding this comment.
Fixed in 8f284a1. A fallback candidate is now accepted only after OpenTypeFont.Parse and the coverage HbShaper both succeed; failures are caught and the candidate is skipped. Test: A_fallback_font_that_does_not_parse_is_skipped (corrupt maxp).
| if (FollowsNeighbourFont(cp)) font = -1; | ||
| else if (primary.HasGlyph(cp) || char.IsWhiteSpace(concatText[i]) || char.IsControl(concatText[i])) font = 0; | ||
| else font = resolver.FindFallbackFont(style, cp); |
There was a problem hiding this comment.
Fixed in 8f284a1, same change as the Codex thread: one font per extended grapheme cluster, so marks and emoji ZWJ sequences are never split across fonts. Variation selectors and format characters (ZWJ/ZWNJ) are not required in the font's cmap for the cluster to count as covered.
| if (!families.IsDefaultOrEmpty) | ||
| { | ||
| foreach (var family in families) AddCandidate(family); | ||
| } | ||
| AddCandidate(_defaultFamily); | ||
| foreach (var family in SystemFallbackFamilies) AddCandidate(family); |
There was a problem hiding this comment.
Fixed in 8f284a1. The chain now keeps a candidate cursor and resolves candidates lazily, in the fixed order, only until one covers the cluster; indexes are positions among resolved candidates, so they stay stable. An uncoverable cluster walks the rest once, then the answer is memoized. Test: Fallback_candidates_are_resolved_lazily_until_one_covers_the_character.
…eight caps the automatic minimum PR #384 review: - The column content-height floor (content-sized and flex:1 items) is now capped by max-height, in the emission and the BlockLayouter pre-measure. - A definite height no longer switches the automatic minimum off: the floor is min(content, height) (CSS Flexbox 4.5 specified size suggestion). - diagnostics-codes: a break-inside:avoid element taller than a page splits between its children (as the page-break guide says); it is not a forced overflow. Test pins that it splits, keeps every paragraph, and does not report. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…float-adjusted widths PR #385 review: - The measure pass resolves nested boxes' block-axis % padding and % margins against the containing inline size (they are not rewritten to used px until emission, so px reads treated them as 0). - Nested table and multicol measures use the content width resolved by MeasureInlineGeometry (with % padding) instead of re-reading px insets. - In an intrinsic probe, an explicit width's padding is subtracted at the same base the border box was built with. - The outer dispatch passes its float-adjusted available range, so an auto-width block beside a float is measured at its emitted width. Found while testing: the descendant-dominant check compared the deepest child against the parent's whole border box, so a box whose bottom padding + border exceeded its content (one short line in a padding:24px card) lost its bottom padding and the next box overlapped it. It now compares against the content-area bottom, using the deepest child bottom. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…t broken faces, docs - ExtractFace writes Apple's 'true' sfnt signature as 0x00010000, so the safety validator accepts the extracted face (it was indexed but never resolved). - SystemFontResolver walks every candidate of a CSS-generic chain and skips a collection face whose full parse fails, instead of returning null. - The collection sniff loops until 4 bytes are read (partial reads). - Docs: one summary per IsDangerousTableTag overload; the enumerator's class docs describe per-face collection indexing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…, skip unparsable fonts - LineBuilder picks one font per extended grapheme cluster (UAX #29): the first fallback covering ALL of it, else the font covering its base, so a base and its marks (or an emoji ZWJ sequence) are never split. Variation selectors / format characters are optional for coverage. Malformed UTF-16 (a lone surrogate) stays on the primary instead of throwing. - IShaperResolver.FindFallbackFont takes the cluster's codepoints. - HarfBuzzShaperResolver resolves fallback candidates lazily, in fixed order, only until one covers the cluster (stable indexes), and skips a candidate that HarfBuzz can't load or the OpenType parser can't parse. - SyntheticFont.Build(char, char) maps two independent characters (tests). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts: # CHANGELOG.md
Stacked on #386 (task 3). Task 4 of 4.
Problem
Each text run used ONE font: the first
font-familyentry that resolved. A character that font lacks (✔, ▲, →, or text in another script) was drawn as a.notdefbox, even when a later family or an installed symbol font had it. Five tester samples (02, 06, 07, 08, invoice-06) reportedFONT-MISSING-GLYPH-001.Fix
LineBuilder.Shape): a run that shaped to a visible.notdefis split by font coverage. Each piece is shaped with the first font of the style's fallback chain that covers it (ItemizedRun.FontIndex). Combining marks, variation selectors and format characters follow their neighbour's font. Runs with no tofu are untouched (no extra cost).IShaperResolvergainsResolve(style, fontIndex)andFindFallbackFont(style, codepoint), with defaults that mean "no fallback" (test resolvers are unchanged).HarfBuzzShaperResolverbuilds the chain once per (family stack, weight, style): the rest of the author stack, the default family, then a fixed list of symbol / broad-coverage system families. Each entry is validated like the primary, de-duplicated by content hash, and skipped if unusable. A customIFontResolveronly gets fallback fonts it resolves itself.TextPainterembeds glyphs from the run's own font, and uses the primary font's metrics for the baseline, so fallback glyphs sit on the same line.Tests
FontFallbackTests: split by coverage, marks stay with their base, leading mark, uncovered char stays tofu and is reported, resolver without a chain is unchanged, no split when fully covered; chain order (author stack, then system families), de-dupe, built once, unsafe bytes skipped; end to end: two embedded fonts, three text pieces on one baseline, and the diagnostic still fires when nothing covers the character.FONT-MISSING-GLYPH-001is gone from all files. Only the 5 files that had tofu changed. Page counts are unchanged.🤖 Generated with Claude Code