Operational transform libraries for collaborative markdown editing (Elixir + TypeScript) - #56
Closed
calvin-archastro wants to merge 2 commits into
Closed
Operational transform libraries for collaborative markdown editing (Elixir + TypeScript)#56calvin-archastro wants to merge 2 commits into
calvin-archastro wants to merge 2 commits into
Conversation
Elixir library (src/elixir/operational_transform, ArchAstro.OperationalTransform): - TextOperation: retain/insert/delete algebra with apply/invert/compose/ transform (TP1) and cursor transformation; ot.js-compatible wire format; all lengths in Unicode code points - Document + Document.Server: GenServer document authority that serializes submissions, rebases stale ops against history, and broadcasts operations/cursors/presence (Registry + DynamicSupervisor addressing) - Client: pure sync state machine (synchronized / awaiting_confirm / awaiting_with_buffer); Actor: gen_statem editing client built on it - Tests: unit fixtures, 1,500 seeded fuzz iterations of the OT laws, and a concurrent gen_statem actor-fleet convergence simulation Example Phoenix server (examples/ot_example): - doc:<id> channel; acks are routed through the document server (submit_async) so they can never overtake concurrent broadcasts on the websocket — replying synchronously from handle_in reorders and diverges - serves the demo UI and a read-only JSON snapshot API for tests TypeScript library + demo (src/ts/operational_transform): - mirrored TextOperation/Client (shared fixtures pin cross-language behaviour) and a DocSession Phoenix transport - Google-Docs-like editor on CodeMirror 6: markdown rendered as rich text, syntax-highlighted code fences, monospace-aligned tables, inline image widgets, live preview pane, shared cursors/selections, presence, toolbar - vitest suites incl. a seeded multi-client convergence fuzz harness - browser end-to-end test (agent-browser): two real browsers fire colliding edits and must converge with the server byte-for-byte Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Once a markdown table is detected, that region of the document switches modes: the markdown is replaced by an interactive grid widget that behaves like a spreadsheet, while the document of record stays plain markdown — every grid interaction compiles to a minimal text edit against the hidden cell spans, so the OT layer needs no special cases (disjoint spans per cell; same-cell edits merge character-by-character). UX (all verified with real browser events via agent-browser): - type in any cell; text wraps inside the cell - Enter moves down / grows the table from the last row; Tab/Shift-Tab and arrow keys navigate; Escape returns to prose - right-click menu: insert/delete row/column, align column, delete table - drag header borders to resize columns; widths are encoded as delimiter dash counts so they replicate collaboratively - hover add-row/add-column strips; toolbar grid picker inserts new tables - Cmd-Z in cells routes to document-level undo Implementation: - demo/table-model.ts: pure parse/edit layer (cell spans, minimal diffs, row/col/align/width edits) — 15 unit tests round-tripping real markdown - demo/table-widget.ts: block widget with DOM reconciliation, synchronous focus restore across concurrent structural rebuilds (async restore dropped in-flight keystrokes), caret preservation, context menu - browser-test/table-editing.mjs: two-browser e2e covering the full UX plus concurrent cell edits and a row-insert racing header typing, asserting editor A == editor B == server; shared harness extracted - rich-blocks.ts: tables render as widgets; larger parse budget prevents transient raw-markdown flicker after large inserts Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Contributor
Author
|
Built in the wrong repository — this work belongs in firstlanding and has moved to https://github.com/ArchAstro/firstlanding/pull/8335 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
A working prototype of collaborative document editing over operational transformation, in two mirrored libraries plus a Google-Docs-like demo:
src/elixir/operational_transform(ArchAstro.OperationalTransform)TextOperation— retain/insert/delete algebra:apply,invert,compose,transform(TP1) and cursor transformation; ot.js-compatible wire format; all lengths in Unicode code pointsDocument+Document.Server— a GenServer document authority that serializes submissions, rebases stale ops against history, and broadcasts operations/cursors/presence (Registry + DynamicSupervisor addressing)Client— pure sync state machine (synchronized/awaiting_confirm/awaiting_with_buffer);Actor— a gen_statem editing client built on it (the doc authority is plain serialized state → GenServer; the client protocol is a real 3-state machine → gen_statem)src/elixir/operational_transform/examples/ot_example— minimal Phoenix server:doc:<id>channel, static demo serving, and a read-only snapshot API. Acks are routed through the document server (submit_async) so they can never overtake concurrent broadcasts on the websocket — replying synchronously fromhandle_inreorders and diverges (found by the browser test).src/ts/operational_transform— mirroredTextOperation/Client(unit fixtures share literals with the Elixir suite to pin cross-language behaviour, including the insert tie-break), aDocSessionPhoenix transport, and the demo editor: CodeMirror 6 rendering markdown as rich text — styled headings/emphasis, syntax-highlighted code fences, inline images, live preview pane, shared cursors/selections, presence, sync-status pill, formatting toolbar.Spreadsheet-mode tables
Once a GFM table is detected, that region switches modes: the markdown is replaced by an interactive grid that behaves like a spreadsheet — type in cells (text wraps), Enter grows the table from the last row, Tab/arrows navigate, right-click for Excel-style row/column/align/delete commands, drag header borders to resize columns (widths encode as delimiter dash counts, so they replicate collaboratively), hover add-row/add-column strips, and a Docs-style toolbar grid picker to insert tables. The document of record stays plain markdown: every grid interaction compiles to a minimal text edit on hidden cell spans, so the OT layer needs no special cases.
Test plan
mix test(library): 49 tests — unit fixtures, 1,500 seeded fuzz iterations of the OT laws (TP1/compose/invert/codec/cursor bounds), and a concurrent gen_statem actor-fleet convergence simulationmix test(example): 6 channel tests incl. two editors submitting concurrently against the same revision over the real wire protocolnpm test: 44 tests — mirrored unit fixtures, a 170-scenario multi-client convergence fuzz harness, and the pure table-model layer round-tripping real markdownnpm run test:browser(needsagent-browser+ Elixir): two end-to-end suites, each driving two isolated real browsers against a live server — colliding text edits, and the full table UX (cell typing, Enter-adds-row, right-click column insert, drag-resize, row-insert racing header typing) — assertingeditor A == editor B == serverbyte-for-byte plus marker survivalAll suites green locally; both browser suites passed repeatedly.
🤖 Generated with Claude Code