Feature: Option to disable forwarding messages into rabbitmq

[sansmoraxz]

Is your feature request related to a problem? Please describe.

When MQTT brokers don't have any consumers in deployment, it feels unnecessary to publish into it.

Describe the feature you are requesting, as well as the possible use case(s) for it.

Refer above.

Striving for minimal resource utilization. 5k message per second stress test resulted in mem usage of rabbitmq around 6 GB. Meanwhile NATS and other go services were chilling at around 50-300 MB utilization. (Although granted it was done in a low power CPU laptop)

Indicate the importance of this feature to you.

Should-have

Anything else?

No response

[drasko]

Very interesting, and a very good idea. We should definitely do that.

However, take into account that NATS and other services chill with so small utilization, is because they are all Go-native. Only outlier here is Rabbit (with Erlang BEAM underneath). Because all these messages forwarded to Rabbit also go through NATS and all other adapters (bridge brokers them to all adapters, especially WS and CoAP).

What you could try, for the sake of experiment, is replace in your deployment Rabbit with another NATS, set for MQTT - it is doable as in SuperMQ MQTT brokers are in-place replacable (like any other protocol adapter, they are decoupled by mGate), so you can bring your own.

[dborovcanin]

Helllo @sansmoraxz, forwared is used to forward messages from internal message broker (NATS) to MQTT broker. Did I understand that you do not need such a protocol bridge? Do you also want to have MQTT messaging or you want to completely turn it off?

[sansmoraxz]

For now completely turn off.

If need later may want to enable again via some env flag.

[drasko]

@dborovcanin, he wants to disable MQTT forwarding when there are no subscribers on the MQTT broker (no MQTT clients). It should be enabled at the moment when the first subscriber comes.

[sansmoraxz]

Would it be possible to make the mqtt broker itself optional?

We are constrained to our live devices relying on mqtt, but on cloud side we dont have such protocol limitations.

So mqtt adapter can become just an abstraction layer into our application stack.

[drasko]

Of course. It is just a matter of deployment - all adapters are optional.

[drasko]

@arvindh123 and @dborovcanin can help with that, the forwarder should also be disabled, so I would prefer that they confirm this and explain the changes, but I think just removing MQTT broker in the docker-compose file should do the job.

[dborovcanin]

This is true. MQTT can be removed/commented from the deployment; other protocols will work.

[dborovcanin]

Returning to the original premise - When MQTT brokers don’t have any consumers in deployment, it feels unnecessary to publish to them. If there are no consumers, the brokers are likely unnecessary and can simply be commented out in the Compose file (i.e., not deployed).

Regarding the idea of disabling MQTT forwarding when there are no subscribers, and enabling it once the first subscriber connects, that approach is risky. Its correctness depends heavily on how both the broker and clients are configured, and on whether clients expect to receive only live messages or also historical data. In any case, I don’t think this kind of behavior should be handled automatically.

I think we can close this issue, marking "commenting out MQTT broker (rabbitmq in compose) and adapter (mqtt-adapter in compose)" as a solution.

[sansmoraxz]

I think we can close this issue, marking "commenting out MQTT broker (rabbitmq in compose) and adapter (mqtt-adapter in compose)" as a solution.

OK that works out. Kinda need mqtt brokers ocassionally though. Swapped to mosquitto.

However I am thinking would it be possible to have a seperate adapter option that works as a MQTT to NATs bridge (not use any MQTT brokers)?

Something like:
https://github.com/tada/mqtt-nats

[drasko]

That would make SuperMQ too dependent on NATS broker. The central broker in SMQ is also in-place replaceable. It can be done easily, though, with the config - we tried it and it works nicely.

Regarding Mosquitto - please have in mind that this is not a scalable broker, suitable for cloud deployments. It is single-instance only. SMQ targets global scale, millions of connected devices - Mosquitto is not a candidate here.

[sansmoraxz]

That would make SuperMQ too dependent on NATS broker. The central broker in SMQ is also in-place replaceable. It can be done easily, though, with the config - we tried it and it works nicely.

Ohh! That's cool. It was my understanding that NATS was a central broker for the entire SuperMQ stack. So is it correct to say that it can be swapped into Iggy with just a few changes?

[sansmoraxz]

We have been looking for a good scalable MQTT broker that does not take up a lot of resources.

Regarding Mosquitto it's good enough for our present test scenario of around 10k things, periodically publishing messages.

For high scalablity something we are considering is wind-c/comqtt. But some additonal work would be needed here too.