diff --git a/docs/docs/design/playground-simulation-phase-2.md b/docs/docs/design/playground-simulation-phase-2.md index b3e62d7..d75b95b 100644 --- a/docs/docs/design/playground-simulation-phase-2.md +++ b/docs/docs/design/playground-simulation-phase-2.md @@ -1,21 +1,30 @@ # Playground simulation — Phase 2 -Status: **planned** — the staged plan for everything -[Phase 1](playground-simulation.md) deferred. Phase 1 proved one call between two -services on the real `@comline/runtime`. Phase 2 turns that into a place to -reason about *distributed* behaviour: many services and connections, an -unreliable wire, controllable time, user-written behaviour, and — last — the -actual generated module running in the browser. Nine milestones (**2a–2i**), -each independently reviewable and shippable, each landing with its acceptance -check green. Still almost entirely app code over the vendored runtime; the one -`comline-core` touch (a codegen re-export for route A) is called out in -[Open questions](#open-questions). Affects `ComlineProject/playground`, then -`ComlineProject/docs` (the tutorial embeds the result). +Status: **in progress** — 2a–2f and the full engine port are done, as PRs #1–#10 +against [`ComlineProject/simulator`](https://github.com/ComlineProject/simulator). +Remaining: 2g (framing / codec matrix), rewiring the playground UI onto the new +engine, and 2i (the tutorial embed). 2h is **subsumed** — see below. + +Phase 1 proved one call between two services on the real `@comline/runtime`. +Phase 2 turns that into a place to reason about *distributed* behaviour: many +services and connections, an unreliable wire, controllable time, user-written +behaviour, and the shareable, replayable session. + +> **Architecture changed mid-phase (2026-09).** Phase 1–2a–2e were built as +> TypeScript inside `ComlineProject/playground`, over a *vendored copy* of +> `@comline/runtime` plus a re-implementation of the generated client / dispatch +> glue. That grew large enough to be its own thing. The engine is now +> [`ComlineProject/simulator`](https://github.com/ComlineProject/simulator) — a +> Rust crate → WASM — and the playground / tutorial / docs are thin hosts over +> its `Sim` `wasm-bindgen` surface. This retires the vendored port, the +> re-implementation, and the drift guard, and it collapses milestone 2h. The +> sections below are updated to match; [Architecture](#architecture-the-engine-is-its-own-crate) +> is new. ## Goal -Phase 1's sentence was *two services generated from the same schema can talk, -and refuse to when they were not*. Phase 2's is: +Phase 1's sentence was *two services generated from the same schema can talk, and +refuse to when they were not*. Phase 2's is: > the same schema, run as a **system** — many services, a wire that drops and > delays, time you can step through — behaves the way the protocol says it @@ -27,373 +36,271 @@ Concretely, by the end of Phase 2 the playground can: one client talking to several servers, and a **gateway** node that is a client of one protocol and a server of another, forwarding calls; - inject **faults** per connection — drop, delay, reorder, corrupt, partition — - and see how a generated client copes (timeouts, retries where the schema has - them, `RuntimeError` surfacing); -- **control time** — step / pause / play / speed — over a virtual clock, so a - race is inspectable frame by frame; + and see how a client copes (timeouts, `RuntimeError` surfacing); +- **control time** — step / advance / play — over a virtual clock, so a race is + inspectable frame by frame; - **record and replay** a session, and share it as a URL; -- run a **user-written behaviour** for a function in a Worker sandbox instead of - the six canned ones; +- run a **user-written behaviour** for a function as a sandboxed Rhai script + instead of only the canned ones; - compare the **same call across framings and codecs** (datagram / JSON-RPC, JSON / MessagePack) side by side; -- run **route A** — transpile the generated TypeScript in-browser and use the - real `Client` / dispatcher as an instance's implementation, with route B - still the default and the drift guard still watching; - be **embedded** in a tutorial lesson with a fixed, partly-locked topology. +## Architecture — the engine is its own crate + +The simulation engine is +[`ComlineProject/simulator`](https://github.com/ComlineProject/simulator), a Rust +crate compiled to WASM. The host (playground, tutorial, docs) provides the UI and +drives it through the `Sim` `wasm-bindgen` surface — construct from a compiled +shape, edit the topology, drive calls, read the frame log, record / replay. + +**Why a separate crate.** + +- It builds against **`comline-runtime` directly** (the `alloc` tier — `no_std`, + allocation-free, synchronous). No vendored port of the TS runtime, no + re-implementation of the generated client / dispatch glue, no drift guard. +- The synchronous contract makes the engine a **discrete-event simulation**: one + event queue whose time *is* the virtual clock. A call schedules a + request-delivery event and settles later; there are no promises to interleave, + so the 2c / 2d timing races the TS version fought simply cannot occur. +- Being a *separate* WASM module from the playground's editor wasm means the + schema crosses the boundary as bytes regardless. It crosses as the **`Shape` + JSON** the editor's `describe_project` already emits — a small, stable + projection — rather than by linking `comline-core` (which would put a second + copy of the compiler in the bundle). See `src/shape.rs`. + +**Module layout** (`ComlineProject/simulator/src/`): + +``` +rng.rs seeded PRNG (mulberry32), bit-for-bit with the JS reference +faults.rs the unreliable-wire spec + transforms +frame.rs the frame tap the inspector reads +format.rs the JSON WireFormat +shape.rs the compiled-project projection (describe_project mirror) +clock.rs the virtual clock + its event queue +wire.rs one connection's tapped, fault-injecting channel +behavior.rs the 8 server behaviours (reply … forward, script) +generic.rs a dispatcher driven by a ProtocolShape, no codegen +model.rs the Session: nodes, instances, connections, ops +session_codec.rs the Session ⇄ #s=… shareable link +record.rs record & replay +engine.rs many connections over one clock; the discrete-event pump +framedecode.rs a raw frame → the inspector's decoded view +facade.rs the #[wasm_bindgen] Sim surface +``` + +**Scripting is a cargo feature** (`script`, default on) — it pulls Rhai, which +~5× the wasm (445 KB → ~2.1 MB; ~165 → ~580 KB gzipped). `--no-default-features` +keeps the lean build for the tutorial embed. See the simulator repo's `README` +for the current lazy-load thinking. + ## What changes from Phase 1 -Phase 1's [architecture](playground-simulation.md#architecture) holds. Three -core generalisations carry the rest: +Three core generalisations carry the rest: | Phase 1 | Phase 2 | |---|---| -| `Session.connection: Connection \| null` | `Session.connections: Connection[]` — the engine holds a `Map` | +| `Session.connection: Connection \| null` | `Session.connections: Connection[]` — the engine holds one live wire per connection, diffed against the session | | node ≡ instance | a **node** hosts one or more instances; an instance still belongs to exactly one node | -| `TappedTransport` with a fixed `latencyMs` | `TappedTransport` driven by a `FaultSpec` + a `Clock` — delivery is scheduled, not `setTimeout`-d | -| one merged frame log | frames carry `connId`; the log filters/splits by connection | -| behaviour = one of six canned fns | behaviour = canned **or** a sandboxed user snippet, same `Behavior` interface | -| route B only | route B default; route A opt-in per instance | - -No change to `describe_project`, the vendored runtime contract, or the drift -guard. `model.ts` / `engine.ts` / `transport.ts` grow; `generic.ts` and -`behavior.ts` gain, not change. - -New module layout (added to Phase 1's `app/src/sim/`): - -```text -sim/ - clock.ts virtual clock + scheduler; real-time and stepped modes - faults.ts FaultSpec, the per-frame decision (drop/delay/reorder/corrupt) - topology.ts Node, multi-instance, the connections[] operations - record.ts session ⇄ URL, input capture, replay driver - sandbox/ - host.ts Worker lifecycle, the call protocol, timeout/kill - guest.ts runs in the Worker: eval the snippet, expose the ctx API - routea/ - transpile.ts esbuild-wasm wrapper: generated .ts → an ES module blob - load.ts import the module, adapt Client / dispatcher to Behavior/engine - embed.ts mount(el, { schemas, topology, locked, lesson }) — the tutorial entry -``` +| transport with a fixed `latencyMs` | a tapped `Channel` driven by a `FaultSpec` + the clock — delivery is a scheduled event, not `setTimeout` | +| one merged frame log | frames carry a connection id; the log filters / splits by connection | +| behaviour = one of six canned fns | behaviour = one of seven canned fns **or** a sandboxed Rhai script, same `Behavior` trait | ## Milestones -| # | Step | Acceptance | -|---|---|---| -| **2a** | **Many connections** — `connections[]`, N live wires, canvas draws & selects N edges, frame log gains a connection column + per-connection filter | fan-out: one `Chat` server, two clients, each calls `send`, both replies correct, the log shows two connections; removing one connection leaves the other live | -| **2b** | **Nodes host many instances + forwarding** — a node with a `client` of `A` and a `server` of `B`; a **Forward** behaviour that calls out on another connection before replying | a `gateway` node relays `B.send` → `A.send` → back; the frame log shows the nested call on the second connection; a cycle is refused | -| **2c** | **Fault injection** — `FaultSpec` per connection: `dropProb`, delay distribution, `reorderWindow`, `corruptProb`, `partition` toggle; inspector controls; log annotations | with `dropProb=1` on responses a `send` call surfaces `RuntimeError("timeout")` after the client's window; clearing the fault, the next call succeeds; a corrupted frame shows `framing: undecodable` | -| **2d** | **Virtual clock** — scheduler over a `Clock`; step / pause / play / speed; deterministic given a seed | with a 200 ms delay fault and the clock paused, `send` is issued, "step" advances one frame at a time, the reply lands only after time is advanced past 200 ms; same seed ⇒ same frame order twice | -| **2e** | **Record & replay + shareable session** — `Session` (topology, behaviours, faults, seed) ⇄ URL; input capture; replay driver | a recorded 3-call session replays to a byte-identical frame log; the URL round-trips a two-node fan-out and reconnects it on load | -| **2f** | **User-written behaviours** — Worker sandbox; a function runs `(params, ctx) => Outcome` with a timeout + a small API; canned behaviours stay | a snippet returning `{ ok: { body: params.text.toUpperCase() } }` drives `send`; an infinite-loop snippet is killed and the call gets `RuntimeError`; the snippet cannot reach `window` / `fetch` | -| **2g** | **Framing / codec matrix** — the same call through datagram + JSON-RPC, JSON + MessagePack, side by side; `MsgPackCodec` | one `send` rendered four ways; the decoded bodies match; the JSON-RPC and datagram request frames differ only as their specs say | -| **2h** | **Route A — run the real module** — `esbuild-wasm` transpiles the generated `.ts`; it loads as an ES module; the generated `Client` / dispatcher runs against a browser build of `@comline/runtime` as an instance's implementation; route B stays default | an instance flipped to "route A" serves `send` from the *generated* `ChatDispatcher`; its frames are byte-identical to route B's for the same behaviour (the drift guard, now live in the UI); transpile failure falls back to route B with a notice | -| **2i** | **Embeddable ``** — `mount(el, { schemas, topology, locked, lesson })`; fixed topology, some controls locked; no header / edit view | the tutorial's "two services talk" lesson embeds the sim with `chat-1` / `chat-2` pre-wired and the palette hidden; a second lesson embeds a fan-out | - -2a–2e are the topology-and-time spine and should land in order. 2f (sandbox) -and 2g (matrix) are independent and can interleave. 2h (route A) depends only on -2f's Worker plumbing being available (it runs the transpiled module in a Worker -too) and should come after the drift guard is exercised by the matrix in 2g. -2i is last — it packages a stable surface. +| # | Step | Acceptance | Status | +|---|---|---|---| +| **2a** | **Many connections** — `connections[]`, N live wires, canvas draws & selects N edges, per-connection frame log | fan-out: one `Chat` server, two clients, both replies correct, log shows two connections; removing one leaves the other live | ✅ | +| **2b** | **Nodes host many instances + forwarding** — a gateway node; a **Forward** behaviour that calls out on another connection before replying | a gateway relays `B.send` → `A.send` → back; the log shows the nested call; a cycle is refused | ✅ | +| **2c** | **Fault injection** — `FaultSpec` per connection: drop / delay / reorder / corrupt / partition; a client call timeout | `dropProb=1` on responses → the call times out and the wire goes `dead`; clearing the fault + `rebuild` restores service; a corrupted frame reads `framing: undecodable` | ✅ | +| **2d** | **Virtual clock** — the discrete-event queue; step / advance / play; deterministic given a seed | a delay fault + stepped clock: `send` is issued, `step` fires one event at a time, the reply lands only after time passes the delay; same seed ⇒ same frame order twice | ✅ | +| **2e** | **Record & replay + shareable session** — `Session` ⇄ `#s=…` link; input capture; replay | a recorded session replays to a byte-identical frame log; a link round-trips a fan-out and reconnects it | ✅ | +| **2f** | **User-written behaviours** — a sandboxed **Rhai** script as an 8th behaviour kind; the canned ones stay | a script returning `#{ body: params[0] }` drives `send`; an infinite-loop script is stopped (operations limit), not hung; `state` persists between calls | ✅ | +| **2g** | **Framing / codec matrix** — the same call through datagram + JSON-RPC, JSON + MessagePack, side by side | one `send` rendered four ways; the decoded bodies match; the JSON-RPC and datagram request frames differ only as their specs say | ▫ next | +| **2h** | ~~Route A — transpile the generated TS in-browser~~ | — | **subsumed** — the Rust engine already runs the real `comline-runtime` contract; there is no re-implementation to reconcile | +| **2i** | **Embeddable ``** — a fixed, partly-locked topology; no header / edit view; the lean (`--no-default-features`) wasm | the tutorial's "two services talk" lesson embeds the sim with `chat-1` / `chat-2` pre-wired and the palette hidden | ▫ | + +Also on the list before Phase 2 closes: **rewire the playground UI** — replace +`app/src/sim/*.ts` with a thin view over `Sim`, and delete the vendored +`@comline/runtime` + the drift-guard test. ## Topology (2a–2b) ### The model -```ts -interface Node { - id: string; - label: string; // "gateway", "chat-1" - x: number; y: number; - instanceIds: string[]; // ≥ 1 -} - -interface Connection { - id: string; - clientId: string; // an instance id - serverId: string; // an instance id - faults: FaultSpec; // 2c; identity (no-op) until then -} - -interface Session { - shape: ProjectShape; - nodes: Node[]; - instances: Instance[]; - connections: Connection[]; - seed: number; // 2d -} -``` +`Session` holds `nodes`, `instances`, `connections`, plus `latencyMs`, +`callTimeoutMs`, `seed`, and a `clockMode` preference. A `Node` is a canvas box +with `instanceIds` (≥ 1). A `Connection` is `{ clientId, serverId, faults }` +between two instances. An `Instance` keeps its Phase 1 fields (`role`, +`schemaNs`, `protocol`, `behaviors`, `irHash`) and gains `nodeId`. The id +counters live *in* the session (not module globals), so a decoded link keeps +allocating fresh ids without collision. -An `Instance` keeps its Phase 1 fields (`role`, `schemaNs`, `protocol`, -`behaviors`, `irHash`) and gains `nodeId`. `addInstance` either makes a fresh -node or, when dropped onto an existing node, appends to it — a node badge shows -the count. +`shape` is dropped from the serialized form and recomputed from the open schemas +on load; the behaviour map serializes in a stable key order. ### Connection rules - client and server ends are **instances**, opposite roles, same - `schemaNs::protocol` (unchanged from Phase 1). -- an instance may be **one end of many connections**: a server with N client - connections is fan-out; a client with N server connections is fan-in. The - engine builds one `duplex()` + serve loop **per connection** — a server - instance in three connections runs three serve loops over three taps, which is - what three real peers would cause. + `schemaNs::protocol`. +- an instance may be **one end of many connections** — fan-out / fan-in. The + engine runs one tapped channel + dispatcher per connection. - **no duplicate** `(clientId, serverId)` pair. -- a **cycle** through forwarding is refused at call time (see below), not at - connect time — the topology graph is only a cycle once behaviours forward - along it. +- a **cycle** through forwarding is refused at call time, not at connect time. ### The engine -`engine.connectAll(session)` replaces `connect`. It diffs the desired -`connections[]` against the live `Map`: opens new ones, -closes removed ones, rebuilds ones whose framing / `irHash` / faults changed. -Each `LiveConnection` keeps its Phase 1 shape plus `connId`. `GenericClient.call` -is now reached as `live(connId).call(fn, params)`; the call form picks the -connection when the selected client has more than one. +`engine.sync(session)` diffs `session.connections` against the live wire set: +opens the ones that appeared, closes the ones that vanished, leaves the rest +running. `engine.rebuild(session)` closes everything and re-opens — for a schema +edit, a latency change, or a replay — re-seeding the fault RNG so a stepped run +from `session.seed` is reproducible. Each wire records a real 31-byte handshake +frame each way and refuses (`connectionError == "handshake"`) on an IR-hash +mismatch — the version-skew demo, decided directly rather than via an async +exchange. -### Forwarding (2b) +A call doesn't block: `engine.call(connId, fn, params)` frames the request, +schedules its delivery, and returns a request id; the outcome lands later and +`engine.result(id)` reads it (`ok` / `err` / `undecodable` / `timeout`). -A **Forward** behaviour on `(serverInstance, fn)`: - -```ts -{ kind: "forward", - config: { viaConnectionId: string; targetFn: string; mapParams?: string /* jsonata-lite, optional */ } } -``` +### Forwarding (2b) -`run(ctx)` resolves the `LiveConnection` for `viaConnectionId` (which must have -this instance's **node** as its client end), calls `targetFn` with the params -(mapped if `mapParams` is set), and returns the downstream outcome as its own — -an `ok` becomes an `ok`, a `SimRemoteError` becomes an `err` with the same -ordinal. The frame log shows the downstream `Call` / `Reply` on the second -connection, indented under the first. A forward that would re-enter a connection -already on the call stack fails with `RuntimeError("forwarding cycle")` — the -stack is carried on the `BehaviorCtx`. +A **Forward** behaviour on `(serverInstance, fn)` carries +`{ viaConnectionId, targetFn }`. When dispatched it yields a `Forward` step; the +engine relays the call on `viaConnectionId` and, when that inner call settles, +answers the outer one with its outcome — an `ok` stays `ok`, an `err` keeps its +ordinal. A `forwarding` set carries the connections currently mid-relay; a +forward that re-enters one is refused with a `forwarding cycle` error. There is +no parked stack frame — the outer request is answered from a continuation keyed +by the inner call's request id. -This is enough for a **gateway / proxy** demo: a node that is -`server of Public` and `client of Internal`, forwarding `Public.request` to -`Internal.handle`. +This is enough for a **gateway / proxy** demo: a node that is `server of Public` +and `client of Internal`, forwarding `Public.request` to `Internal.handle`. ## Faults (2c) -```ts -interface FaultSpec { - dropProb: number; // 0..1, per frame, direction-filterable - delay: { min: number; max: number }; // ms, uniform; 0..0 = none - reorderWindow: number; // hold up to N frames and release shuffled; 0 = ordered - corruptProb: number; // 0..1; flips a random byte in the body - partition: boolean; // hard cut both directions until cleared - applyTo: "requests" | "responses" | "both"; -} -``` - -Lives on the `Connection`; the engine hands it to both `TappedTransport`s of -that connection. `TappedTransport.send` becomes: **record the frame** (always, -with a `fault` annotation), then ask the `FaultSpec` + the seeded RNG what to do -— deliver now, deliver after a delay via the `Clock`, hold for reorder, corrupt -then deliver, or drop. A dropped frame is in the log greyed with `dropped`; a -corrupted one decodes to `framing: undecodable` and the receiving runtime raises -`framing` / `serialization` as it would for real. - -Partition is the coarse control the tutorial wants for "what happens when the -network splits": every frame both ways is dropped, pending `call`s time out, -and clearing it does **not** replay held frames (a real partition loses them). - -Inspector: a **faults** section per selected connection with the five controls; -a connection with any active fault draws its edge dashed-amber. +`FaultSpec` on the `Connection`: `dropProb`, `delayMin` / `delayMax`, +`reorderWindow`, `corruptProb`, `partition`, `applyTo` +(`requests` / `responses` / `both`). The engine hands it to the connection's +`Channel`; the inspector edits it in place with no reconnect. + +`Channel.send` records the frame (always, with a `fault` annotation), then decides +against the spec + the seeded RNG: deliver now, deliver after a delay scheduled on +the clock, hold for reorder, corrupt then deliver, or drop. RNG draws are +consumed in a fixed order so a stepped run is reproducible. A dropped frame is in +the log annotated `dropped`; a corrupted one decodes to `framing: undecodable`. +Partition cuts every frame both ways (handshakes included) and does not replay +held frames when lifted. + +**Timeout / dead.** A non-one-way call schedules a timeout at +`callTimeoutMs`. If no reply lands, the call settles `timeout` and the wire goes +`dead` — every later call on it fails fast until `engine.rebuild`. A forwarded +inner call that times out propagates an error to the outer, and both wires die. +The clock's `schedule` returns a handle so a call that settles early cancels its +pending timeout (otherwise a far-future timeout would keep the sim "busy" and +over-advance virtual time). ## Time (2d) -Phase 1 delivers with `setTimeout`. Phase 2 routes every delayed delivery -through a `Clock`: +There is one clock — a virtual time value plus a `(due, seq)`-ordered event +queue, generic over the event payload. There is no real-time / stepped *engine* +split; the host decides how fast to advance: -```ts -interface Clock { - now(): number; - after(ms: number, fn: () => void): () => void; // returns a cancel - mode: "real" | "stepped"; -} -``` - -- **real** — `after` is `setTimeout`; `now` is `performance.now()`. The default; - identical to Phase 1 behaviour. -- **stepped** — a priority queue of `(dueAt, fn)`. "step" pops the earliest and - runs it; "play" drains at `speed × wall time`; "pause" stops draining. `now` - is the virtual time, advanced to each entry's `dueAt` as it fires. +- **drain** (`run`) — pop events until the queue is empty; the idiom for a test + or a settled call. +- **advance(ms)** — fire every event due within the window (including ones + scheduled while firing), then park time at the edge; the + `requestAnimationFrame` / playback path. +- **step** — fire the single earliest event. -The engine, the fault delays, and the sandbox timeout all take the same `Clock`. -Determinism: in stepped mode with a fixed `seed`, the fault RNG and the queue -order are fully determined, so a session's frame log is reproducible — the basis -for record & replay. - -Controls sit in the frames header: `⏸ ▶ ⏭` and a speed select. A stepped clock -with pending entries shows a count (`3 events queued`). +`clockMode` on the `Session` records the user's preference for the UI; the engine +is always stepped. Determinism: a fixed `seed` + the queue order fully determine +a session's frame log — the basis for record & replay. ## Record & replay, shareable sessions (2e) -**Serialisation.** `Session` minus `shape` (which is recomputed from the -schemas) serialises to compact JSON → deflate → base64url → the URL fragment. -On load, if a `#s=` fragment is present and the current schemas produce a -matching set of `ir_hash`es, the topology is restored and connected; on a hash -mismatch the instances load **stale** (Phase 1's resync flow applies). +**Link.** `Session` minus `shape`, wrapped in a `{ v: 1, session }` envelope → +JSON → base64url → the `#s=` fragment. On decode the current shape is +re-injected and the id counters are lifted past everything loaded; instances keep +their stored `irHash`, so a schema that has moved on since the link was made +loads them *stale* and the resync flow applies. Scalars default leniently +(`callTimeoutMs` 3000, `seed` 1, `real` mode) so a hand-made link still decodes. -**Recording** captures the ordered list of user inputs — `call(connId, fn, -params)`, behaviour edits, fault edits, clock steps — each stamped with the -virtual `now`. **Replay** loads the session, forces `clock.mode = "stepped"`, -and feeds the inputs back at their timestamps. With the same seed the frame log -is byte-identical; this is the acceptance check and a regression guard for the -engine. +**Recording** captures the ordered user inputs — a call, a behaviour edit, a +fault edit — each stamped with the clock time it happened at, relative to +record-start, plus the session link at record-start. -The frames header gains `● rec` / `▷ replay`. A recording exports as a JSON file -and imports back. +**Replay** decodes the snapshot, forces stepped mode, rebuilds, and for each +event advances to its time, applies the input, and drains what it scheduled. The +discrete-event clock makes this deterministic by construction — no +`Promise.allSettled` / microtask scaffolding. The frame log is byte-identical +across runs; that equality is the engine's regression guard. ## User-written behaviours (2f) -A seventh behaviour kind, **Script**: +An 8th behaviour kind, **Script** — `{ kind: "script", config: { source } }` +where `source` is a Rhai program. -```ts -{ kind: "script", config: { source: string } } -``` +In scope: `params` (the decoded request) and `state` (a map that persists between +calls on this instance). The last expression is the reply; `throw` raises an +error (ordinal 0, `{ error: }`). A one-way function still sends nothing. +`state` survives a `rebuild` while the instance's `schemaNs::protocol` does, same +as the canned behaviour configs. -`source` is an ES module body that default-exports -`async (params, ctx) => Outcome`, where `Outcome` is Phase 1's -`{ ok } | { err: { ordinal, data } } | { none: true }` and `ctx` exposes only: - -```ts -interface ScriptCtx { - fn: FnShape; // the function being served (read-only) - proto: ProtocolShape; - state: Record; // persists across calls on this instance - sleep(ms: number): Promise; // via the engine Clock - log(...args: unknown[]): void; // to the frame log's side channel -} -``` - -**Sandbox.** One `Worker` per scripted instance. `host.ts` posts -`{ params }`; `guest.ts` `import()`s a blob URL of the source once, caches the -export, runs it, posts back the `Outcome` or an error. No `window`, `fetch`, -`XMLHttpRequest`, `WebSocket`, or dynamic `import` of anything but the initial -blob — enforced by running the guest with those globals shadowed to `undefined` -and a CSP on the worker. A call that exceeds `scriptTimeoutMs` (clock-driven) -terminates the Worker and returns `RuntimeError("timeout")`; the Worker respawns -for the next call. +**Sandbox.** Rhai has no file / network / process API to reach for, so the +sandbox is about bounding work and memory: `max_operations` (200k), +`max_call_levels`, `max_expr_depths`, and string / array / map size caps; +`print` / `debug` are silenced; the engine is built `no_module` (no script +`import`). An infinite loop hits the operations limit and errors — it does not +hang the sim. A script that does not compile errors at run time with the compiler +message. This is why Rhai was chosen over a JS-snippet-in-a-Worker: safe by +construction, and no cross-thread call protocol. -The inspector shows a small code editor (reuse the CodeMirror setup already in -the app) with the `ScriptCtx` type surfaced as a doc comment. A syntax error is -reported inline and the behaviour falls back to **Drop** until fixed. +Behind the `script` cargo feature. Without it the variant still exists but its +factory returns a stub that reports scripting is off, and the wasm stays lean. -Not in scope: importing packages, multiple files, TypeScript types on the -snippet (it is plain JS). Those are Phase 3 if wanted. +Not in scope: importing modules, multiple files, a type-checked script surface. ## Framing / codec matrix (2g) -A **compare** panel: pick a client, a function, params once, and the engine runs -the call over a throwaway connection in each of `{datagram, jsonrpc} × -{json, msgpack}`, showing the four frame sets in columns with a shared body view. -It asserts the decoded request params and the decoded reply are equal across all -four; divergence is a bug in a framing or codec. - -`MsgPackCodec` implements the vendored `Codec` interface (`name: "msgpack"`, -`encode`/`decode`) with a small dependency-free MessagePack pair vendored -alongside the runtime. It does **not** need `comline-typescript` to gain -MessagePack — the sim's codec is its own; a note goes in the runtime repo that a -real one would live there. - -## Route A — run the real module (2h) - -The playground already generates the TypeScript (`generate_project`, the -"generated" tab). Route A runs it. - -1. **Transpile.** `esbuild-wasm` (loaded once, ~3 MB, lazy) compiles the - generated `.ts` — client + dispatcher + the schema's types — to one ES - module, rewriting the `@comline/runtime` import to a blob URL of a **browser - build of the vendored runtime** (the runtime is dependency-free ES already; - this is a bundling step, not a port). -2. **Load.** `import(blobUrl)` gives the real `ChatClient`, `serveChat`, - `ChatDispatcher`. -3. **Adapt.** For a **server** instance on route A, `serveChat(impl, transport, - codec, framing)` replaces `GenericDispatch` — `impl` is built from the - instance's behaviours (a canned behaviour becomes a generated-signature method; - a **Script** behaviour is called through the same sandbox). For a **client** - instance, `ChatClient` replaces `GenericClient`. -4. **Guard.** The Phase 1 drift guard becomes a **live** check: with an instance - on route A and its peer on route B and the same behaviour, the frame logs - must stay byte-identical. A mismatch is surfaced in the UI, not just CI. - -Route A is opt-in per instance (a toggle in the inspector), defaults off, and -falls back to route B with a notice if transpilation fails or `esbuild-wasm` -will not load. It is the riskiest milestone — if `esbuild-wasm` size or -cross-origin-isolation requirements make it impractical on GitHub Pages, 2h -ships as "route A behind a flag, not in the default bundle" and the rest of -Phase 2 stands without it. - -## Embeddable `` (2i) +A **compare** view: pick a client, a function, params once, and run the call over +a throwaway connection in each of `{datagram, jsonrpc} × {json, msgpack}`, +showing the four frame sets side by side with a shared body view. It asserts the +decoded request params and the decoded reply are equal across all four; +divergence is a bug in a framing or a codec. -`embed.ts` exports: +This lands in the crate: a JSON-RPC `Framing` alongside `DatagramFraming`, and a +MessagePack `WireFormat` alongside the JSON one. The `Shape` already carries a +protocol's `framing`; the engine's per-wire framing becomes a small enum instead +of hard-wired datagram, and `framedecode` gains the JSON-RPC path (its +`DecodeCtx` already takes the framing). -```ts -mount(el: HTMLElement, opts: { - schemas: FileInput[]; // fixed for the lesson - topology?: SerializedSession; // pre-wired nodes / connections - locked?: ("palette" | "connections" | "behaviours" | "faults" | "schemas")[]; - lesson?: string; // analytics / deep-link id -}): { destroy(): void } -``` - -It mounts the canvas + inspector + frame log without the header or the edit -view, honours `locked` (a locked control renders read-only), and never touches -the URL fragment (the host page owns that). The tutorial's runtime-demo lesson -embeds it with `chat-1` / `chat-2` pre-wired and `palette` + `schemas` locked; -a later lesson embeds a fan-out with `connections` unlocked so the reader adds -the third client. +## Embeddable `` (2i) -This is the point where `sim/` gets a stable public surface and a short -`README`; until 2i it is internal to the app. +The tutorial mounts the sim with a fixed topology and some controls locked, no +header or edit view, and never touching the URL fragment. Because the engine is +already a packaged crate with a `Sim` facade, "embed" is mostly a host-side view: +the tutorial links the **lean** (`--no-default-features`) wasm, ships a +precompiled `Shape` and a session link for the lesson, and renders a cut-down +canvas + inspector + frame log over it. ## Open questions -- **Serve-loop cost at fan-out** — one server instance in N connections runs N - serve loops. Fine for the ~single-digit N a demo shows; note a ceiling and a - friendly error past it rather than letting the tab hang. -- **`esbuild-wasm` on GitHub Pages** *(blocks 2h's default-on)* — size (~3 MB) - and whether it needs `SharedArrayBuffer` / cross-origin isolation, which - Pages does not set. Fallback: `sucrase` (smaller, strips types only, no - bundling) with a hand-written import shim, or route A stays flag-only. -- **`comline-core` touch for route A** — the transpile step needs the generated - files *and* their intended import graph. `generate_project` already returns - the file set; confirm it also returns enough for the `@comline/runtime` - import rewrite (it should — the import is a fixed string). If a re-export or - a manifest field is missing, that is the one small core/codegen change in - Phase 2. -- **Reorder + virtual clock interaction** — a reorder window holds frames until - N accumulate or a timeout; in stepped mode "timeout" is virtual, so a held - frame needs a queue entry. Confirm the release rule reads cleanly when time is - paused (probably: release on step if the window is non-empty and no more - frames are pending). -- **Script `state` across a rebuild** — a schema edit rebuilds instances; does a - scripted instance keep its `state`? Lean yes while `schemaNs::protocol` - survives, same as behaviour configs. -- **Record format stability** — the replay guard only holds if the record - format and the engine's input handling stay in lockstep; version the record - JSON and refuse a mismatched one rather than replaying it wrong. - -## Risks and cuts - -- If 2h (route A) proves impractical, Phase 2 still delivers 2a–2g + 2i and the - drift guard stays CI-only — the sim remains a faithful re-implementation, just - not the literal module. -- 2d (virtual clock) is the highest-leverage / highest-churn change — it touches - every delayed path. If it slips, 2c ships with real-time-only faults and 2e's - replay is "best effort, not byte-identical" until 2d lands. -- The sandbox (2f) is a security surface. Keep the guest globals denylist and - the worker CSP under test; a snippet reaching the network is a release - blocker. +- **Serve cost at fan-out** — one server instance in N connections runs N + dispatchers over N taps. Fine for the single-digit N a demo shows; note a + ceiling and a friendly error past it. +- **Scripting wasm size** — ~580 KB gzipped for the `script`-on build. Accepted + for now. The playground rewire decides: default on, or ship lean and lazy-load + the scripted wasm when a `Script` behaviour is selected. The tutorial always + builds lean. +- **Playground rewire scope** — how thin the TS view actually gets. Everything + stateful (topology, engine, clock, recorder) moves behind `Sim`; the TS keeps + the canvas, the inspector forms, the frame-log rendering, and the CodeMirror + editor for scripts. ## Not in this phase -- Package imports / multi-file / TypeScript in Script behaviours. -- A real MessagePack codec in `comline-typescript` (the sim vendors its own). -- Length-prefixed stream transport (Phase 1's `duplex()` is message-oriented; - a byte-stream transport with its own framing is Phase 3). +- Module imports / multi-file / a typed surface in Script behaviours. +- A length-prefixed byte-stream transport (the channel is message-oriented). - Persisting sessions server-side; only the URL fragment and file export. - More than one schema project open at once in the sim.