Skip to content

Run the unmodified iroh crate relay-only on wasip2, relaying over polymorph-websocket - #24

Closed
lann wants to merge 4 commits into
udp-wake-probefrom
iroh-relay-ws-spike
Closed

Run the unmodified iroh crate relay-only on wasip2, relaying over polymorph-websocket#24
lann wants to merge 4 commits into
udp-wake-probefrom
iroh-relay-ws-spike

Conversation

@lann

@lann lann commented Aug 6, 2026

Copy link
Copy Markdown
Collaborator

The spike #14 and #18 point at, stacked on #18 (base = udp-wake-probe; retarget to main when #18 merges). Upstream iroh v1.0.3 — the crate, not a port — runs as a wasm32-wasip2 component under jco/JSPI, relay-only, against a stock iroh-relay server, with the relay websocket owned by the host through the polymorph-websocket host module (the same browser-first JS that serves the WIT package) and delivered to the guest as synthetic datagrams, so the unmodified tokio reactor parks and wakes exactly as in #18's probes.

Shape

  • guest/: two iroh endpoints in one component (presets::Minimal, RelayMode::custom, clear_ip_transports, clear_address_lookup). B dials A by (EndpointId, relay URL); 50 echo rounds × 512B on one bi stream. QUIC (noq + rustls/ring, RFC 7250 raw public keys) runs end-to-end in-guest; the relay never holds connection keys.
  • vendor/: cargo-vendored sources with the wasi-enablement patches — the whole inventory:
    • noq-udp: route wasi to posix_minimal + non-vectored recv_from (no recvmsg in wasi-libc) + AsFd From-impl — functional single-datagram UDP over wasi:sockets, not a stub;
    • netwatch: route wasi to its netdev-free posix_minimal branch (2 lines);
    • iroh-dns: tolerate the missing system resolver config (1 cfg arm);
    • iroh-relay: the real patch — a wasm_wasi connect() mirroring the browser branch, with DatagramPipe replacing WsBytesFramed (one datagram = one ws message = one relay frame; control preamble carries URL + subprotocols to the bridge at 127.0.0.1:1; in-band challenge-response handshake, ExportKeyingMaterial → None like the browser);
    • iroh: untouched except short-circuiting the net_report HTTPS relay probe on wasi (home-relay selection gates on it; no HTTP client path exists; the websocket dial itself proves reachability — upstream's browser build passes this probe only because reqwest becomes fetch there).
  • host/: shim descended from Wake a parked tokio reactor from JS through a synthetic wasi:sockets shim #18's (multi-socket registry, wasi:random/random, filesystem/ip-name-lookup stubs, RUST_LOG passthrough) + bridge.mjs (datagrams ⇄ Websocket from .deps/websocket/js/jco/websocket.js).

Measured (Node 24.18, loopback --dev relay, stable across runs)

metric value
dial through relay (incl. QUIC handshake) 23–30ms
echo RTT over relay p50 ~0.5ms
echo RTT p90 / max 1.0–1.9ms / 7–11ms (first round carries stream setup)
guest component size 7.3MB (opt-level=s + lto)
ws messages per run ~180 each way, all accounted for

Known cosmetic issue: after done, teardown logs a burst of relay_recv_channel closed errors (socket actor polls while the relay transport unwinds) — shutdown ordering, not a functional defect.

What this establishes for #14

The "relay leg" seam question is answered concretely: the patch surface to run the real iroh crate relay-only on a browser-shaped host is four small vendored patches plus one honest probe short-circuit, with the polymorph-websocket host module unmodified in its package role. Combined with #18's wake-path result, the browser leg of the fat-guest route is now demonstrated end-to-end at the relay tier; what remains is the WebRTC custom transport (datagram-shaped by construction) and the identity/webcrypto split.

Verified: just iroh-relay-ws (cold relay start + reuse paths, multiple runs).

…ymorph-websocket

The spike issue #14 and PR #18 point at: upstream iroh v1.0.3 as a
wasm32-wasip2 component under jco/JSPI, relay connectivity only, with the
relay websocket owned by the host through the polymorph-websocket host
module — the same browser-first JS that serves the WIT package — and
delivered to the guest as datagrams on a synthetic UDP socket, so the
unmodified tokio reactor parks and wakes exactly as in the udp-wake
probes. iroh-relay's wasi branch replaces WsBytesFramed with a
DatagramPipe (one datagram = one ws message = one relay frame) behind a
tiny control protocol to the bridge at 127.0.0.1:1; the in-band
challenge-response handshake and everything above it — relay frames, QUIC
(noq + rustls/ring, RFC 7250 raw public keys), streams — runs unmodified
in-guest against a stock iroh-relay server.

Vendored wasi-enablement patches (cargo-vendored sources, iroh untouched
except one probe): noq-udp routes wasi to posix_minimal with a
non-vectored recv_from (functional single-datagram UDP over wasi:sockets,
not a stub); netwatch routes wasi to its netdev-free posix_minimal
branch; iroh-dns tolerates the missing system resolver config; iroh-relay
gains the wasm_wasi connect(); iroh short-circuits the net_report HTTPS
relay probe on wasi (no HTTP client path exists; the websocket dial
itself proves reachability), because home-relay selection gates on it.

Measured (Node 24.18, two endpoints in one component, stock relay,
--dev, loopback): dial-through-relay 23-30ms; echo RTT over the relay
p50 ~0.5ms, p90 1.0-1.9ms (50 rounds, 512B); guest component 7.3MB
(opt-level=s + lto). Shutdown logs an expected burst of
relay_recv_channel-closed errors after done (teardown ordering), which
is cosmetic.

Manual recipe (research probe, not a ci gate): just iroh-relay-ws.
lann added 2 commits August 6, 2026 08:36
The wasip2 enablement this spike wrote into vendor/noq-udp turns out to
have landed upstream ten days ago as n0-computer/noq#773 (merged
2026-07-27), structurally identical — posix_minimal alias, non-vectored
recv_from, AsFd-based UdpSockRef construction — plus one fix this
spike's compile-only usage could not catch: set_nonblocking must go
through std (ioctl FIONBIO) rather than socket2 (fcntl F_SETFL), which
WASI rejects. The vendored copy is now a verbatim backport of #773, and
this patch disappears entirely once iroh consumes a noq-udp release
containing it.
The noq-udp wasi fix is merged upstream but unreleased (the published
1.1.1 was cut from a pre-merge base), so the vendored backport gives way
to a [patch.crates-io] git pin. Released noq 1.1.1 requires noq-udp
^1.1.1 while the git tree identifies as 1.1.0, so a lone noq-udp patch
would split the graph (noq keeping the unpatched registry copy); noq and
noq-proto ride the same rev to keep one coherent noq-udp. Verified: the
spike runs unchanged on the pinned rev (connect 25ms, echo p50 472us).

The other four vendored patches have no upstream counterpart to pin
(checked at HEAD: netwatch is still espidf-only, iroh-dns has no wasi
arm, iroh-relay and iroh have no wasi handling).
@lann

lann commented Aug 6, 2026

Copy link
Copy Markdown
Collaborator Author

Vendored noq-udp replaced by a git pin (6145165): the wasi fix turned out to be already upstreamn0-computer/noq#773 (merged 2026-07-27, structurally identical to this spike's patch plus a set_nonblocking fix ours lacked: socket2 uses fcntl(F_SETFL), which WASI rejects; std uses ioctl(FIONBIO)). The published noq-udp 1.1.1 was cut from a pre-merge base (publish date 2026-07-29 postdates the merge — red herring), so the fix exists in no resolvable release; hence [patch.crates-io] pinning noq/noq-proto/noq-udp to the same rev — released noq 1.1.1 requires noq-udp ^1.1.1 while the git tree identifies as 1.1.0, so a lone noq-udp pin would split the graph and keep the unpatched registry copy on noq's edge. Spike unchanged on the pinned rev (connect 25ms, echo p50 472µs).

Audited the remaining vendored patches against upstream HEAD: none exist upstream — netwatch's posix_minimal alias is still espidf-only, iroh-dns has no wasi arm, iroh-relay and iroh have no wasi handling. Those four stay vendored; netwatch's is the obvious next upstream PR in #773's mold.

Also checked (empirically) the tempting shortcut of RUSTFLAGS="--cfg posix_minimal" instead of patching: it flips rustc-side module routing but cannot remove a Cargo dependency edgeposix_minimal lives in two languages (rustc cfgs and Cargo target-tables, which can't reference build.rs aliases), so netdev stays in the unit graph and still fails on IFF_*; and for noq-udp the alias was only one of #773's four hunks. Ambient-cfg scoping hazards aside, the per-crate definition-site patches are the correct shape — which is also the shape upstream itself chose.

The released crates lag upstream too far to keep chasing (noq#773
merged but unreleased, netwatch restructured on main, iroh main carrying
relay-path fixes like #4444's reconnect-backoff handling). The guest now
declares iroh as a git dependency pinned to main (735958f8), overridden
via [patch] with vendor/iroh — the same rev re-vendored plus the
net_report probe short-circuit. Its manifest's own path deps pull the
sibling vendor/iroh-relay and vendor/iroh-dns (same rev, same spike
patches); iroh-base resolves from crates.io, byte-identical to the
release at this rev. Every file this spike patches is unchanged between
v1.0.3 and main, so the patches ported by copy.

Verified end to end against the stock v1.0.3 relay server (a
cross-version interop datum in itself): connect 27ms, echo p50 508us,
guest 7.2MB.
@lann

lann commented Aug 6, 2026

Copy link
Copy Markdown
Collaborator Author

Now tracking iroh main (7245a3c): the guest declares iroh = { git, rev = 735958f8 } with a [patch] on the git source pointing at vendor/iroh — the same rev re-vendored plus the net_report probe short-circuit. The vendored manifest's own path deps pull the sibling vendor/iroh-relay (datagram-pipe connect) and vendor/iroh-dns (resolver-config arm) at the same rev; iroh-base resolves from crates.io, byte-identical to the release at this rev.

Porting cost was zero code: every file this spike patches is unchanged between v1.0.3 and main (total non-test drift across the three crates: two Cargo.toml lines + the relay actor's reconnect-backoff fix, n0-computer/iroh#4444 — which this spike now inherits). The netwatch situation on main is a restructure (dedicated netdev cfg alias, 'keep in sync with the Cargo.toml gate'), not wasi support — the vendored 0.19.1 patch stays, and the future upstream PR should target the new alias shape.

Re-verified against the stock v1.0.3 relay server — a cross-version interop datum in itself (main client ↔ 1.0.3 relay): connect 27ms, echo p50 508µs, guest 7.2MB. Vendor tree shrank by ~12.5k lines (releases' bundled locks/tests dropped by cargo-vendor's git flattening).

@lann lann closed this Aug 6, 2026
@lann
lann deleted the iroh-relay-ws-spike branch August 6, 2026 14:31
@lann

lann commented Aug 6, 2026

Copy link
Copy Markdown
Collaborator Author

Superseded by the long-lived experiment branch upstream-iroh (this branch, renamed — the rename auto-closed this PR). Continued in #25.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant