Slice shaped text by cluster instead of guessing a range - #24
Closed
mantissaman wants to merge 1 commit into
Closed
Conversation
Rendered text duplicated letters. Suite came out as SSuite, financial as fifinancial, allocated as allocatedd and 78701 as 778701. The text itself was always correct, so this only ever showed up once glyphs were positioned. Line breaking cuts a shaped segment into chunks at break opportunities, and each chunk needs its share of the already-shaped glyphs. It picked them by slicing the glyph array with character indices, which is only right when shaping happens to be one glyph per character. When it was not, it fell back to estimating a proportional range with rounding, and rounding at both ends let neighbouring chunks claim the same glyph. The glyph was then drawn twice and a letter appeared doubled. One ligature was enough to poison a whole run. The character count and the glyph count only have to disagree once for every chunk in that run to take the estimating path, which is why letters doubled in words with no ligature anywhere near them. Shaping already knows the answer. Each glyph carries a cluster, the byte it came from, so ShapedText and TextSegment now keep it and chunks select the glyphs whose cluster falls in their byte range. Nothing is estimated. A cluster belongs to the chunk holding its first byte, so a ligature spanning a break lands wholly on one side and is drawn once. Sub-segment clusters are rebased so a chunk can be split again. Page-one PNGs change for letter, contract, invoice, quote and report. Verified by rendering letter and invoice before and after: the doubled letters are gone and nothing else moved. Baseline updated with a reason. Closes #23.
Contributor
Author
|
Closing this without merging to main. The work is not dropped, it has moved to Nothing about the change itself has altered. Same fix, same tests, same regenerated baselines, and the branch passes the full suite at 364 tests with the harness matching. The reasoning in the description above still stands and is worth keeping for whoever reviews the release branch. In short: glyphs are now selected by shaper cluster rather than by a proportional guess, so neighbouring chunks can no longer claim the same glyph and draw a letter twice. #23 stays open until the release branch lands. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #23.
The bug
Line breaking cuts a shaped segment into chunks at break opportunities, and each chunk needs its share of the glyphs that were already shaped for the whole segment. It picked them like this:
Slicing by character index is only right when shaping is one glyph per character. When it is not, the estimate took over, and rounding independently at each end let neighbouring chunks claim the same glyph. That glyph was drawn twice, and a letter appeared doubled.
One ligature was enough to poison a whole run. The character count and the glyph count only have to disagree once for
glyph_ids.len() == total_charsto be false for the entire segment, so every chunk in that run took the estimating path. That is why letters doubled in words with no ligature anywhere near them.The fix
Shaping already knows the answer and we were throwing it away. Every glyph carries a cluster, the byte offset it came from.
ShapedTextandTextSegmentnow keep it, and a chunk selects the glyphs whose cluster falls inside its byte range. Nothing is estimated.Two details:
The old character-index path survives only as a fallback for segments with no cluster data, such as generated tab leaders, and it no longer guesses a range.
Result
letter.png, before and after:SSuite 400Suite 400eenthusiasmenthusiasmfifinancialfinancialddetectiondetectionallocateddallocatedsuub-100mssub-100msensurreensurecentralizinggcentralizingAustin, TX 778701Austin, TX 78701th e foundationthe foundationI rendered
letterandinvoicebefore and after and read them line by line. Every doubling is gone and nothing else moved.Output delta
Five fixtures change:
letter,contract,invoice,quoteandreport. That is the fix, not a regression, and it is why the doubling has been invisible to the harness until now. Baseline updated with an audit reason.Checks
cargo fmt --all --check,cargo clippy --workspace --all-targets -- -D warningsand the full suite all pass. 364 tests, up from 361.Three tests, aimed at the property rather than the symptom. One splits a ligature-bearing segment at every byte boundary and asserts no glyph is ever emitted twice and none is lost. One asserts a ligature spanning a break lands on one side only. One asserts sub-segment clusters index into their own text.
Breaking
ShapedTextandTextSegmentboth gain a public field.