Maximum number of consumers

[Mierunski]

I have a use case of message queue for millions of endpoints which have to receive message and process it.
We are using main jetstream with op.* subject.
Each endpoint is using its own subject op.ABCDF12345, when endpoint connects to server its reading all messages intended for its subject, process them and respond.
Every endpoint can connect and disconnect at any time, thats why we create_or_get consumer per each endpoint moment they are connecting to server. However if endpoints connect in bursts (eg. 2000 endpoints per second) it results it timeouts on consumer/subscription creation from server.

We wanted to keep separate consumers to have dedicated message queues per each endpoint, as messages can arrive while endpoint is offline and they need to be processed when they come back online.

From other information, we know beforehand which endpoints exist, so maybe there is option to prepare beforehand somehow for this?

What is actual number of consumers which NATS Jetstream is aimed for? If it can handle millions, is there some technique which could speed up initial allocation of them? Will scaling nats cluster increase speed of consumer allocation?

[wallyqs]

do you plan the consumers be memory only R1 or R3 and are they durable or ephemeral? In current main branch there are optimizations to reach millions of consumers by increasing the meta_compact_size to larger values, but overall it would be better if you shard the streams so that you do not have more than 20K consumers or so per stream.

Being able to create consumers at rates higher than 2K per second on a single cluster is something that we are actively working on, you could can monitor whether the consumer create rate is falling behind by monitoring the size of the WAL in raftz, if it keeps increasing that is when your consumer creates will start getting increase latency.

[Mierunski]

Plan was to go with R3 durable, to leave deliverability handling to NATs completely, flow is

• we send messages to n endpoints
• whenever they come online they respond
• we only listen for responses and update state accordingly
  With single consumer per endpoint, its automatically managed by nats and proven to work good with smaller numbers od endpoints.
  Current approach to solve this issues that we tested is single consumer for whole stream and dividing messages with tokio internal communications. While performance looks good, deliverability suffers as we have to have internal queues per every endpoint and it adds burden on our side to persist them somehow in case server crashes, so we are not fully happy with this solution.

Is there something else which can improve this? Will sharding of streams improve consumer creation rate as well?