Skip to content

feat(redis): support Redis Cluster via comma separated seed nodes - #1318

Open
bakiburakogun wants to merge 1 commit into
nextcloud:mainfrom
bakiburakogun:feat/redis-cluster-support
Open

feat(redis): support Redis Cluster via comma separated seed nodes#1318
bakiburakogun wants to merge 1 commit into
nextcloud:mainfrom
bakiburakogun:feat/redis-cluster-support

Conversation

@bakiburakogun

Copy link
Copy Markdown

Fixes #1317

Summary

RedisAdapter.createRedisClient() builds a single-node client with createClient(), which does not follow MOVED redirections. Pointed at a Redis Cluster it connects successfully, the Socket.IO Redis Streams adapter sets up, and then roughly half of all operations fail at run time depending on which slot a key hashes to — a silent, partial failure rather than a clean one.

This treats a comma separated REDIS_URL as a list of cluster seed nodes:

STORAGE_STRATEGY=redis
REDIS_URL=redis://node1:6379,redis://node2:6379,redis://node3:6379

A single URL behaves exactly as before. createCluster() comes from the redis package already in dependencies, so there is no new requirement.

Credentials given on the first seed are passed as cluster defaults, because cluster discovery reports the remaining nodes without auth and they would otherwise be rejected.

Testing

Against a 3-master / 3-replica cluster on a Debian 12 host, writing 40 keys:

client successful failed
createClient() pointed at one node (before) 17/40 23/40 — MOVED 8308 127.0.0.1:7002
createCluster() with three seeds (after) 40/40 0

Reads afterwards returned 40/40.

I also ran the real websocket server from this branch against that cluster, with STORAGE_STRATEGY=redis and three seeds, and connected two Socket.IO clients with valid JWTs to the same board. Both received init-room, room-user-change and user-joined for each other, so room state and the Streams adapter both work through the cluster client.

One rough edge I did not fix

With a cluster client, the first command issued before the slot map is loaded throws rather than being queued, and I saw exactly one such error at startup:

Failed to write heartbeat: TypeError: Cannot read properties of undefined (reading 'master')
    at RedisClusterSlots.getClient (.../cluster/cluster-slots.js:108:59)

It happened once, did not recur, and nothing downstream was affected — the server served boards normally afterwards. The cause is that ServerService starts the connection without awaiting it (this.redisClient.connect().catch(...)), which a single-node client tolerates because it queues commands. Making startup await the connection would fix it properly, but that changes the constructor's shape, so I left it out of this PR rather than restructure startup on your behalf. Happy to follow up with that if you would like it.

Note for administrators

Redis Cluster only has database 0, so the /database_number suffix cannot be used to separate whiteboard keys from other users of the same cluster. I mentioned this in the README next to the cluster example.

The README recommends Redis for multi-node websocket deployments, but the
client is built with createClient(), which does not follow MOVED
redirections. Pointed at a Redis Cluster it connects and then fails roughly
half of all operations at run time, depending on which slot a key hashes to.

Treat a comma separated REDIS_URL as a list of cluster seed nodes and build a
cluster client from it. Credentials given on the first seed are applied to the
nodes discovered afterwards, since cluster discovery reports them without auth.
A single URL keeps its existing behaviour.

Signed-off-by: Baki Burak Öğün <63836730+bakiburakogun@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support Redis Cluster in the websocket server

1 participant