fix(chat): don't apply hard-break newlines to AI-streamed messages

[FelipeRamos-neuro]

Summary

Fixes #384

• The chat message renderer (src/chat-sidebar.tsx:891) applied a blanket .replace(/\n/gi, ' \n') hard-break transform to every rendered message, not just user input as the adjacent comment intended.
• Since this is a raw string transform applied before markdown parsing, it injected trailing double-spaces into every line of AI-streamed fenced code blocks — invisible on screen, but present in the DOM text, corrupting copied code with trailing whitespace (breaking whitespace-sensitive languages like Python, or tripping lint/format diffs after paste).
• Fix: scope the hard-break transform to msg.from === 'user' only, matching the comment's stated intent. AI-streamed messages already contain well-formed markdown and don't need the hack.

See #384 for the full root-cause writeup and reproduction.

Test plan

• Full Jest suite passes (29 suites / 371 tests, no regressions)
• Verified via isolated repro against the pinned react-markdown@9.0.1 + remark-gfm@4.0.0: AI-message code blocks no longer carry trailing whitespace on any line, while user-message hard breaks are preserved
• Manual verification in a running JupyterLab chat panel (not done — no way to launch the dev UI in this environment)

🤖 Generated with Claude Code

[pjdoland]

Thanks for this, and nice root-cause writeup in #384. The trailing-whitespace-in-code-blocks bug is real and worth fixing: injecting \n before markdown parsing does corrupt copied code.

I think the sender-based scoping trades one bug for another, though, because the real axis here is code-fence context rather than who sent the message:

1. Regression for AI prose. MarkdownRenderer uses only remarkGfm (no remark-breaks), so once the hard-break transform no longer applies to AI messages, single newlines in AI output become CommonMark soft breaks and collapse to spaces. Plain-text AI output that separates lines with single newlines (not blank lines or list syntax) now renders run-on: First point\nSecond point shows as First point Second point on one line, where it used to be two. It is lower-frequency since most model output is well-formed markdown, but it is a real visible change, and it would not show up in the code-block repro.

2. The same bug still hits user-pasted code. The .replace still runs on user messages, so if a user pastes a fenced code snippet, every line still gets trailing double-spaces. Scoping by sender only fixes the AI direction.

Both of these go away if the transform is markdown-aware instead of a pre-parse string replace. The cleanest fix is to drop the .replace entirely and add remark-breaks to remarkPlugins in markdown-renderer.tsx alongside remarkGfm. remark-breaks converts soft breaks to hard breaks at the AST level, so it respects code fences (no corruption for AI or user code) while preserving intended line breaks in all prose. That resolves the run-on regression and the user-code case in one move, and removes the string hack. It does add remark-breaks as a dependency, but it is a standard, tiny remark plugin.

Would you be up for switching to that approach? Happy to help if anything is unclear.

[mbektas]

@FelipeRamos-neuro can you please address @pjdoland 's comments above

[FelipeRamos-neuro]

Addressed — reworked the fix per @pjdoland's suggestion: dropped the pre-parse .replace hack and switched to remark-breaks in remarkPlugins. It converts soft breaks to hard breaks at the AST level, so it's fence-aware and fixes both directions (AI-streamed code blocks and user-pasted code), without the AI-prose run-on regression the sender-scoped approach introduced. Rebased onto latest main and pushed.

[mbektas]

@pjdoland could you take another look?

[pjdoland]

This is a correct fix for a real bug, and the approach is the right one.

The old .replace(/\n/gi, ' \n') ran as a raw string transform before markdown parsing, so it could not tell prose from a fenced block and appended two spaces to every line of every code block the model streamed. I reproduced it against the same remark/rehype pipeline the app uses:

input:            x = 1 \n y = 2   (inside a ```python fence)
old (pre-parse):  "x = 1  \ny = 2  \n"
new (remark-breaks): "x = 1\ny = 2\n"

So anyone copying a snippet out of the chat was picking up invisible trailing whitespace on every line. remark-breaks converts soft breaks at the AST level, after fences are parsed, which fixes the corruption without giving up the newline behavior that the hack existed to provide.

I checked the part that worried me most, which is whether removing the transform regresses user-typed messages. It doesn't, and it's worth spelling out why: user prompts are pushed as ResponseStreamDataType.Markdown (chat-sidebar.tsx, where the new message is appended with from: 'user'), so they render through the same case as AI content at the single MarkdownRenderer call site. The plugin sits on that shared renderer, so both senders keep hard-break behavior. Comparing old and new output on paragraphs with single newlines, ordered lists, and GFM tables gives byte-identical HTML; only the fenced and indented code cases differ, which is exactly the bug.

Also confirms the earlier scoped-by-sender iteration was the wrong shape, as the commit message says: scoping by sender would have left user-pasted code corrupted while collapsing AI prose, since no remark-breaks was registered at that point. This supersedes it correctly.

Verified on the branch:

• tsc --noEmit clean
• prettier --check clean
• jest: 30 suites, 376 tests passing
• yarn install --immutable succeeds, so the lockfile is consistent for CI
• remark-breaks@4.0.0 is the right major for react-markdown@^9: both build on unified@^11, and remark-breaks pulls mdast-util-newline-to-break@2, so the plugin runs on the same AST the renderer parses

Looks good to me.

[pjdoland]

@mbektas LGTM. Review above.