Features proposal: replica query support, routing lookup, and password-only AUTH

[va17]

Hello! Thank you for maintaining ered.

I'm an engineer at Riot Games, and we use ered in some of our Erlang services. We maintain a small set of downstream changes for our use cases that we'd like to propose to you here.

Before opening pull requests, I wanted to check whether contributions in the following areas would fit the project's scope and direction:

• Support for issuing commands to cluster replicas
• Routing lookup for multi-node queries, allowing callers to group keys by their Redis Cluster primary
• Password-only AUTH support

If these changes are of interest, we're happy to open separate, focused PRs for each one, including rationale, tests, and documentation, rather than proposing them as a single larger change.

If any of these features are outside the project's intended scope, that's completely understandable. We can continue maintaining them downstream.

Thanks for your guidance, and for maintaining the project.

[zuiderkwast]

Hi Vitor,

nice to hear you're using ered. Note that we might still change the API – it's not stable yet. We'll probably change the default values for config options.

• Support for issuing commands to cluster replicas

This is wanted. I guess we add a config option for that. When it's used, no write commands can be called on that ered instance. If mixing reads and writes is necessary, then those reads should go to primaries, to make sure we get read-after-write correctly.

Does this match what you have in mind?

• Routing lookup for multi-node queries, allowing callers to group keys by their Redis Cluster primary

You mean commands like MGET foo bar where foo and bar are on different nodes? The problem with that is that the command can't be atomic anymore. We could add some option for it, but I don't want that enabled by default. Atomicity is more important. A more complicated case is SUNIONSTORE dest s1 s2 s3 – how would you handle that?

Actually I'm leaning towards not allowing it. We can leave this up to the application to create one command per cluster slot. Howerver, we could add some helper functions to group keys per slot or something like that, to help build up the commands to send. Keys in the same cluster slot are allowed in the same command.

• Password-only AUTH support

This is not necessary. Just use "default" as the username. AUTH default xxxxx is equivalent to AUTH xxxxx.

[va17]

Hi @zuiderkwast , thanks for the detailed response! Apologies for the delay to get back at you.

For replica reads: yes, that matches our use case. Our current downstream implementation exposes this as a separate and straight forward command_replica API, but we’re happy to adapt it to a configuration option and a dedicated read-only ered instance. I agree that writes should not be allowed through that instance, and that applications requiring read-after-write consistency should read from primaries.

For routing: actually a helper to group keys by cluster slot/node is exactly what we had in mind. We are not looking to make ered split commands such as MGET across nodes, or change their atomicity semantics. The application remains responsible for creating independent per-slot commands and combining their results.

Our current downstream helper (in ered_cluster) returns the primary client associated with each key’s slot, which the caller uses as a grouping value:

-spec routing_key(cluster_ref(), key() | [key()]) -> any().
routing_key(ServerRef, Keys) when is_list(Keys) ->
    gen_server:call(ServerRef, {routing_key, Keys});
routing_key(ServerRef, Key) when is_binary(Key) ->
    hd(routing_key(ServerRef, [Key])).

%% ...

handle_call({routing_key, Keys}, _From, State) ->
    RoutingGroup = lists:map(
        fun(Key) ->
            Slot = ered_lib:hash(Key),
            slot_to_client(Slot, State)
        end,
        Keys),
    {reply, RoutingGroup, State}.

slot_to_client(Slot, State) ->
    case binary:at(State#st.slots, Slot) of
        0 -> {error, unmapped_slot};
        Ix -> element(Ix, State#st.clients)
    end.

A helper that returns slot identifiers instead of client references (or directly groups keys by slot) would work for us too.

For password-only AUTH: yeah this one is really a non-issue, we can always set the default username in our configs.

Would you prefer that we start with a focused PR for the read-only replica configuration (or maybe just expose a command_replica for now), and discuss the routing helper’s API separately?

[zuiderkwast]

OK, great!

Yes, separate PRs for each of these, please. I quite like command_replica and I believe ered already maintains connections to all replicas, so it should be quite strait-forward. It's better than my idea.

Regarding routing_key, I'll have to look at the code in context to find out and think about if it's a good API. The high-level direction sounds good.