fix(diff): stop aborting on documents containing wide characters - #20
Merged
Conversation
mdx diff byte-sliced each line to fit the column width, so any character
straddling that offset panicked -- and panic = "abort" turns that into a
SIGABRT with a core dump rather than an error message. Diffing this repo's own
README hit it, on the parity table's checkmarks.
The width handling was wrong even without the crash: it measured with
str::len, which counts bytes, and padded with {:<width$}, which counts chars.
Neither is display width, so a CJK or emoji line put the separator in the
wrong column.
Truncate and pad by display width instead, reusing text.rs. truncate_url was
already exactly this function under a URL-specific name; it is now
truncate_to_width with truncate_url delegating to it.
diff had no tests at all, which is why this survived.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
mdx diffaborts with a core dump on any document containing a wide character. Found while smoke-testing every command against the published v1.5.0 binary.panic = "abort"is set in the release profile, so this is a SIGABRT and a core dump, not a message. Diffing this repo's own README triggers it — the parity table is full of✅.Two bugs, one line apart
The crash:
&s[..max - 3]slices at a byte offset that can land mid-character.The misalignment, present even without the crash: the guard measures
str::len(bytes) while the column padding uses{:<width$}(chars). Neither is display width, so a line of CJK or emoji puts the|separator in the wrong column — a wide character occupies two terminal cells but counts as one char and three bytes.The fix
Truncate and pad by display width, reusing
src/text.rs, which already haddisplay_widthandpad_to_width. It also already had this exact truncation function under a URL-specific name —truncate_urlis nowtruncate_to_width, withtruncate_urldelegating so its callers are untouched.Tests
diffhad no tests at all, which is how this survived. 324 → 331.✅, and mixed emoji — each asserted to exit cleanly--unifiedtruncate_to_width: never splits a character at any width from 0 to 12, counts display columns rather than bytes or chars, marks the cutScanned for siblings
This is the second byte-slicing crash of this class (after
publish.rs), so I swept the source for the pattern. Every other case either takes its index fromfind()— which returns a char boundary — or already goes throughchar_indices().nth().search.rs::truncate_charsandpublish.rs::truncate_descriptionare both correct.diff.rswas the last one.