Skip to content

Slice shaped text by cluster instead of guessing a range - #24

Closed
mantissaman wants to merge 1 commit into
mainfrom
fix/glyph-cluster-slicing
Closed

Slice shaped text by cluster instead of guessing a range#24
mantissaman wants to merge 1 commit into
mainfrom
fix/glyph-cluster-slicing

Conversation

@mantissaman

Copy link
Copy Markdown
Contributor

Closes #23.

The bug

Line breaking cuts a shaped segment into chunks at break opportunities, and each chunk needs its share of the glyphs that were already shaped for the whole segment. It picked them like this:

if seg.glyph_ids.len() == total_chars {
    // slice the glyph array by character index
} else {
    // proportional estimate, with rounding at both ends
    let est_glyphs = (seg.glyph_ids.len() as f64 * byte_frac).round() as usize;
    let glyph_start = (... byte_start as f64 / seg.text.len() as f64).round() as usize;
}

Slicing by character index is only right when shaping is one glyph per character. When it is not, the estimate took over, and rounding independently at each end let neighbouring chunks claim the same glyph. That glyph was drawn twice, and a letter appeared doubled.

One ligature was enough to poison a whole run. The character count and the glyph count only have to disagree once for glyph_ids.len() == total_chars to be false for the entire segment, so every chunk in that run took the estimating path. That is why letters doubled in words with no ligature anywhere near them.

The fix

Shaping already knows the answer and we were throwing it away. Every glyph carries a cluster, the byte offset it came from. ShapedText and TextSegment now keep it, and a chunk selects the glyphs whose cluster falls inside its byte range. Nothing is estimated.

Two details:

  • A cluster belongs to the chunk holding its first byte, so a ligature spanning a break lands wholly on one side and is drawn exactly once.
  • Sub-segment clusters are rebased onto the sub-segment, so a chunk can be split again without offsets pointing outside its own text.

The old character-index path survives only as a fallback for segments with no cluster data, such as generated tab leaders, and it no longer guesses a range.

Result

letter.png, before and after:

before after
SSuite 400 Suite 400
eenthusiasm enthusiasm
fifinancial financial
ddetection detection
allocatedd allocated
suub-100ms sub-100ms
ensurre ensure
centralizingg centralizing
Austin, TX 778701 Austin, TX 78701
th e foundation the foundation

I rendered letter and invoice before and after and read them line by line. Every doubling is gone and nothing else moved.

Output delta

Five fixtures change: letter, contract, invoice, quote and report. That is the fix, not a regression, and it is why the doubling has been invisible to the harness until now. Baseline updated with an audit reason.

Checks

cargo fmt --all --check, cargo clippy --workspace --all-targets -- -D warnings and the full suite all pass. 364 tests, up from 361.

Three tests, aimed at the property rather than the symptom. One splits a ligature-bearing segment at every byte boundary and asserts no glyph is ever emitted twice and none is lost. One asserts a ligature spanning a break lands on one side only. One asserts sub-segment clusters index into their own text.

Breaking

ShapedText and TextSegment both gain a public field.

Rendered text duplicated letters. Suite came out as SSuite, financial
as fifinancial, allocated as allocatedd and 78701 as 778701. The text
itself was always correct, so this only ever showed up once glyphs were
positioned.

Line breaking cuts a shaped segment into chunks at break opportunities,
and each chunk needs its share of the already-shaped glyphs. It picked
them by slicing the glyph array with character indices, which is only
right when shaping happens to be one glyph per character. When it was
not, it fell back to estimating a proportional range with rounding, and
rounding at both ends let neighbouring chunks claim the same glyph. The
glyph was then drawn twice and a letter appeared doubled.

One ligature was enough to poison a whole run. The character count and
the glyph count only have to disagree once for every chunk in that run
to take the estimating path, which is why letters doubled in words with
no ligature anywhere near them.

Shaping already knows the answer. Each glyph carries a cluster, the byte
it came from, so ShapedText and TextSegment now keep it and chunks
select the glyphs whose cluster falls in their byte range. Nothing is
estimated. A cluster belongs to the chunk holding its first byte, so a
ligature spanning a break lands wholly on one side and is drawn once.
Sub-segment clusters are rebased so a chunk can be split again.

Page-one PNGs change for letter, contract, invoice, quote and report.
Verified by rendering letter and invoice before and after: the doubled
letters are gone and nothing else moved. Baseline updated with a reason.

Closes #23.
@mantissaman

Copy link
Copy Markdown
Contributor Author

Closing this without merging to main. The work is not dropped, it has moved to feature/release-0.5.0 as commit 1d845f3, where it is being held for the next release rather than landing on main on its own.

Nothing about the change itself has altered. Same fix, same tests, same regenerated baselines, and the branch passes the full suite at 364 tests with the harness matching.

The reasoning in the description above still stands and is worth keeping for whoever reviews the release branch. In short: glyphs are now selected by shaper cluster rather than by a proportional guess, so neighbouring chunks can no longer claim the same glyph and draw a letter twice.

#23 stays open until the release branch lands.

@mantissaman
mantissaman deleted the fix/glyph-cluster-slicing branch July 29, 2026 23:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Rendered text duplicates letters, most visibly at ligatures

1 participant