Skip to content

fix(diff): stop aborting on documents containing wide characters - #20

Merged
Harsh-2002 merged 1 commit into
mainfrom
fix/diff-multibyte-panic
Sep 1, 2026
Merged

fix(diff): stop aborting on documents containing wide characters#20
Harsh-2002 merged 1 commit into
mainfrom
fix/diff-multibyte-panic

Conversation

@Harsh-2002

Copy link
Copy Markdown
Owner

mdx diff aborts with a core dump on any document containing a wide character. Found while smoke-testing every command against the published v1.5.0 binary.

$ mdx diff notes.md README.md
thread 'main' panicked at src/diff.rs:104:28:
end byte index 35 is not a char boundary; it is inside '✅' (bytes 33..36)
Aborted (core dumped)

panic = "abort" is set in the release profile, so this is a SIGABRT and a core dump, not a message. Diffing this repo's own README triggers it — the parity table is full of .

Two bugs, one line apart

fn truncate(s: &str, max: usize) -> String {
    if s.len() <= max { ... }              // len() is BYTES
    else { format!("{}...", &s[..max - 3]) }   // byte slice → panic
}

The crash: &s[..max - 3] slices at a byte offset that can land mid-character.

The misalignment, present even without the crash: the guard measures str::len (bytes) while the column padding uses {:<width$} (chars). Neither is display width, so a line of CJK or emoji puts the | separator in the wrong column — a wide character occupies two terminal cells but counts as one char and three bytes.

The fix

Truncate and pad by display width, reusing src/text.rs, which already had display_width and pad_to_width. It also already had this exact truncation function under a URL-specific name — truncate_url is now truncate_to_width, with truncate_url delegating so its callers are untouched.

Tests

diff had no tests at all, which is how this survived. 324 → 331.

  • Wide characters across three scripts, 200 consecutive , and mixed emoji — each asserted to exit cleanly
  • Empty files, identical files, --unified
  • Unit tests on truncate_to_width: never splits a character at any width from 0 to 12, counts display columns rather than bytes or chars, marks the cut

Scanned for siblings

This is the second byte-slicing crash of this class (after publish.rs), so I swept the source for the pattern. Every other case either takes its index from find() — which returns a char boundary — or already goes through char_indices().nth(). search.rs::truncate_chars and publish.rs::truncate_description are both correct. diff.rs was the last one.

mdx diff byte-sliced each line to fit the column width, so any character
straddling that offset panicked -- and panic = "abort" turns that into a
SIGABRT with a core dump rather than an error message. Diffing this repo's own
README hit it, on the parity table's checkmarks.

The width handling was wrong even without the crash: it measured with
str::len, which counts bytes, and padded with {:<width$}, which counts chars.
Neither is display width, so a CJK or emoji line put the separator in the
wrong column.

Truncate and pad by display width instead, reusing text.rs. truncate_url was
already exactly this function under a URL-specific name; it is now
truncate_to_width with truncate_url delegating to it.

diff had no tests at all, which is why this survived.
@Harsh-2002
Harsh-2002 merged commit 02aa54f into main Sep 1, 2026
5 checks passed
@Harsh-2002
Harsh-2002 deleted the fix/diff-multibyte-panic branch September 1, 2026 04:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant