[NEW] Support node discovery in standalone (cluster mode disabled)

[asafpamzn]

The problem/use-case that the feature addresses

In cluster mode clusters the clients gets the topology using one of the cluster commands. The ask is to create a similar command for standalone clusters.

It will simplify the usage for users that works in standalone but have dynamic number of replicas. That is, during the life time of the cluster they delete and adds replicas

Description of the feature

Implement new command to get the topology of standalone cluster.

Alternatives you've considered

Migrate to cluster mode enable, but it cannot cover all use cases.

Additional information

Any additional information that is relevant to the feature request.

[hpatro]

It would be nice to capture what are the steps used by the clients currently in standalone setup? Is the information currently built via INFO REPLICATION on the primary ? And what are the shortcomings with it?

[asafpor]

Usually clients just get the endpoints as parameters and do not discover the nodes.

If there is an option to get via replication info we can guide client maintainer to use it. Is the info replication avaliable in elasticache and other valkey providers?

[hpatro]

Thought about it a bit more, I guess what is missing at this point is passing around the hostname information in standalone mode. In cluster mode, we use the cluster bus to gossip that information. if the replica setup was performed via REPLICAOF ip port, it might not be enough information for client to use and connect to.

[asafpamzn]

@hpatro. "Thought about it a bit more,"
is (It == my request for "cluster nodes" in standalone.)
Or (it == INFO REPLICATION)

?

[hpatro]

@asafpamzn around your request and what is lacking.

"I guess what is missing at this point is passing around the hostname information in standalone mode. In cluster mode, we use the cluster bus to gossip that information. if the replica setup was performed via REPLICAOF ip port, it might not be enough information for client to use and connect to."

[asafpor]

@madolson , is that something that we can prioritize.
On one hand we want users to move to CME.
On the other hand. We need this feature in order to support read from replica at CMD. It will allow to client to hold all replicas endpoints.

I think that it is a quick win. See
valkey-io/valkey-glide#3091

[madolson]

Do you have a concrete proposal?

As was mentioned, we have INFO replication. We also discussed exposing CLUSTER SLOTS and CLUSTER SHARDS for standalone installations that know the context. I'm also OK with introducing a new new command that is specific just for standalone.

I think our long term direction should be to fold all the distributed variants of Valkey under the cluster umbrella, but I know @soloestoy has partially opposed that in the past. Can you recommend specifically what you think we should do, then we can discuss in our weekly engineering meeting?

[soloestoy]

but I know @soloestoy has partially opposed that in the past.

@madolson it seems that you've misunderstood me : ). I do not oppose using cluster commands to better present the topology under standalone mode. What I oppose is replacing the standalone with the current cluster mode set up with only one shard, however since the cluster does not support multiple-databases and cross-slot operations, this would prevent standalone users from being able to use it (I thought that's what you meant, and there might be a misunderstanding here as well).

[soloestoy]

I believe our final solution should be unsharded-cluster: it would have only one shard but support multiple-databases, cross-slot operations, and all the functionalities available in standalone mode.

BTW, standalone mode supports chained replication, meaning that a replica can also have its own sub-replicas. This is not supported in the current cluster mode, and it is an important distinction in the topology.

[madolson]

:D Thank you for updating my understanding, I remember we had long discussions about it but I didn't exactly remember the outcome.

I believe our final solution should be unsharded-cluster

I believe in this as well, so I suppose we can build towards that.

[asafpamzn]

From the client side it will make things much easier. Most clients has duplicates classes for cluster and standalone twice the tests...
Can we have for Valkey 9.0 a feature that will support cross slot operations in a cluster with single shard?