Skip to content

Add websocket listener to rendezvous-node (previously dial-only) - #1214

Open
pattayaguy-stack wants to merge 1 commit into
eigenwallet:masterfrom
pattayaguy-stack:fix-rendezvous-node-ws-listener
Open

pattayaguy-stack wants to merge 1 commit into
eigenwallet:masterfrom
pattayaguy-stack:fix-rendezvous-node-ws-listener

Conversation

@pattayaguy-stack

Copy link
Copy Markdown

What was the problem?

Running rendezvous-node behind an nginx + Cloudflare reverse proxy (TLS terminated at the proxy, forwarded to the binary as plain HTTP/ws), every wss connection through it got reset. Direct TCP and the onion transport both worked fine; wss did not.

How did you find/solve it?

create_transport/create_transport_with_onion in swarm.rs build ws_transport.or_transport(tcp_with_dns) (and .or_transport(tor_transport) for the onion variant), so the binary is fully capable of speaking websocket -- but main.rs only ever calls swarm.listen_on() once, with a bare /ip4/0.0.0.0/tcp/{port} address. libp2p-websocket's own listen_on (transports/websocket/src/framed.rs) requires the address to carry a /ws or /wss suffix or it returns MultiaddrNotSupported, so OrTransport always falls through to plain TCP. The ws half of the transport stack was wired up for dialing out (needed to redial the wss entries already in default_rendezvous_points()) but never had a listener of its own to accept anything inbound.

Fix: added a --ws-port flag (default 8889 -- a separate port from --port, since two listeners can't bind the same TCP port) and a second swarm.listen_on() call for /ip4/0.0.0.0/tcp/{ws_port}/ws.

Why this approach?

Kept it to plain /ws, not /wss: this binary has never terminated its own TLS, and the natural place for that stays whatever reverse proxy sits in front of it (nginx + Cloudflare in our case) -- that's also presumably how the existing wss entries in default_rendezvous_points() are run. A separate port rather than trying to share --port avoids touching the existing TCP/onion listen call at all.

Verification

  • cargo fmt -p rendezvous-node -- --check and cargo build --release -p rendezvous-node both clean.
  • Deployed to our own rendezvous node: startup log now shows both listening on address address=0.0.0.0:8888 and listening on address address=0.0.0.0:8889 (protocol /ws).
  • Before this change: a raw websocket Upgrade request straight to the backing port got Connection reset by peer, nginx logged the same on its side.
  • After: ran an isolated DISCOVER query over wss:// straight at our public hostname (Cloudflare -> nginx -> this binary) and got our registered peer back with a live quote, which never happened before.

(Testing/drafting assisted by Claude Code; the bug, fix and verification above are our own.)

The transport stack in swarm.rs builds ws_transport.or_transport(tcp_with_dns)
(and .or_transport(tor_transport) for the onion variant), so the binary is
fully capable of speaking websocket -- but main.rs only ever calls
swarm.listen_on() once, with a bare /ip4/0.0.0.0/tcp/{port} address.
libp2p-websocket's own listen_on (transports/websocket/src/framed.rs)
requires the address to carry a /ws or /wss suffix or it returns
MultiaddrNotSupported, so OrTransport always falls through to plain TCP.
The ws half of the transport stack was wired up for dialing out (needed to
redial the wss entries already in default_rendezvous_points()) but never
had a listener of its own to accept anything inbound.

Found this running our own instance behind an nginx + Cloudflare reverse
proxy (TLS terminated at the proxy, forwarded to the binary as plain
HTTP/ws): every wss handshake through it got reset.

Fix: add a --ws-port flag (default 8889 -- a separate port from --port,
since two listeners can't bind the same TCP port) and a second
swarm.listen_on() call for /ip4/0.0.0.0/tcp/{ws_port}/ws. Kept it to plain
/ws, not /wss -- this binary has never terminated its own TLS, and the
natural place for that stays whatever reverse proxy sits in front of it,
same as before.

Verified on our own deployment: rebuilt, pointed nginx's proxy_pass at the
new port, and ran an isolated DISCOVER query over wss straight at our
public hostname -- it now returns our registered peer, which it never did
before.

(Testing/drafting assisted by Claude Code; the bug, fix and verification
above are our own.)

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant