broker: EVT_CONNECT from all listeners called upon connection

[ymorin-orange]

What hapens:

Given a configuration with multiple listeners, with each listener loading a plugin that registers an EVT_CONNECT callback, when a client connects to a listener, then the EVT_CONNECT callback for all plugins of all listeners are called.

---

How to reproduce:

1. Build the plugins/examples/connection-state/ plugin
2. start a mosquitto broker (with the -v option) with this configuration:

   listener 1883 0.0.0.0
   listener_allow_anonymous true
   
   listener 1884 0.0.0.0
   listener_allow_anonymous true
   plugin /usr/lib/mosquitto_connection_state.so
   
   listener 1885 0.0.0.0
   listener_allow_anonymous true
   plugin /usr/lib/mosquitto_connection_state.so
   
   listener 1886 0.0.0.0
   listener_allow_anonymous true
   plugin /usr/lib/mosquitto_connection_state.so
   plugin /usr/lib/mosquitto_connection_state.so  # Loaded twice, not an error!
   

   • notice that the broker reports loading the plugin for each listener:

     1772517942: Loading plugin: /usr/lib/mosq_test.so
     1772517942: Plugin connection-state has registered to receive 'connect' events.
     1772517942: Plugin connection-state has registered to receive 'disconnect' events.
     1772517942: Plugin connection-state version 1.0 loaded.
     1772517942: Loading plugin: /usr/lib/mosq_test.so
     1772517942: Plugin connection-state has registered to receive 'connect' events.
     1772517942: Plugin connection-state has registered to receive 'disconnect' events.
     1772517942: Plugin connection-state version 1.0 loaded.
     1772517942: Loading plugin: /usr/lib/mosq_test.so
     1772517942: Plugin connection-state has registered to receive 'connect' events.
     1772517942: Plugin connection-state has registered to receive 'disconnect' events.
     1772517942: Plugin connection-state version 1.0 loaded.
     1772517942: Loading plugin: /usr/lib/mosq_test.so
     1772517942: Plugin connection-state has registered to receive 'connect' events.
     1772517942: Plugin connection-state has registered to receive 'disconnect' events.
     1772517942: Plugin connection-state version 1.0 loaded.
     1772517942: Opening ipv4 listen socket on port 1883.
     1772517942: Opening ipv4 listen socket on port 1884.
     1772517942: Opening ipv4 listen socket on port 1885.
     1772517942: Opening ipv4 listen socket on port 1886.
     

3. once the broker is ready to serve, spawn a subscriber on port 1883, that listens to the connection topics: mosquitto_sub -t '$SYS/broker/connection/client/#' -i foo -F '%t %p'
   • notice that there is a first trace for that client, although it connects on a listener that has not loaded the plugin:

     $SYS/broker/connection/client/foo/state 1
     

   • notice that the broker sends a PUBLISH:

     1772517957: New connection from 10.0.2.100:45670 on port 1883.
     1772517957: New client connected from 10.0.2.100:45670 as foo (p4, c1, k60).
     1772517957: No will message specified.
     1772517957: Sending CONNACK to foo (0, 0)
     1772517957: Received SUBSCRIBE from foo
     1772517957:     $SYS/broker/connection/client/# (QoS 0)
     1772517957: foo 0 $SYS/broker/connection/client/#
     1772517957: Sending SUBACK to foo
     1772517957: Sending PUBLISH to foo (d0, q0, r1, m0, '$SYS/broker/connection/client/foo/state', ... (1 bytes))
     

4. run a second MQTT client (it can be whatever, let's just use mosquitto_sub again for the sake of the test): mosquitto_sub -t '#' -i bar
   • notice that there are four events reported by the first subscriber, one per plugin loaded, although this second client connects on a listener that has not loaded the plugin:

     $SYS/broker/connection/client/bar/state 1
     $SYS/broker/connection/client/bar/state 1
     $SYS/broker/connection/client/bar/state 1
     $SYS/broker/connection/client/bar/state 1
     

   • notice also that the broker reports four PUBLISH:

     1772518035: New connection from 10.0.2.100:34908 on port 1883.
     1772518035: New client connected from 10.0.2.100:34908 as bar (p4, c1, k60).
     1772518035: No will message specified.
     1772518035: Sending CONNACK to bar (0, 0)
     1772518035: Sending PUBLISH to foo (d0, q0, r0, m0, '$SYS/broker/connection/client/bar/state', ... (1 bytes))
     1772518035: Sending PUBLISH to foo (d0, q0, r0, m0, '$SYS/broker/connection/client/bar/state', ... (1 bytes))
     1772518035: Sending PUBLISH to foo (d0, q0, r0, m0, '$SYS/broker/connection/client/bar/state', ... (1 bytes))
     1772518035: Sending PUBLISH to foo (d0, q0, r0, m0, '$SYS/broker/connection/client/bar/state', ... (1 bytes))
     

5. terminate that second client with Ctrl-C:
   • notice that there are four events reported by the first subscriber, one per plugin loaded, although this second client connected on a listener that has not loaded the plugin:

     $SYS/broker/connection/client/bar/state 0
     $SYS/broker/connection/client/bar/state 0
     $SYS/broker/connection/client/bar/state 0
     $SYS/broker/connection/client/bar/state 0
     

   • notice also that the broker reports four PUBLISH:

     1772518039: Received DISCONNECT from bar
     1772518039: Client bar [10.0.2.100:34908] disconnected.
     1772518039: Sending PUBLISH to foo (d0, q0, r0, m0, '$SYS/broker/connection/client/bar/state', ... (1 bytes))
     1772518039: Sending PUBLISH to foo (d0, q0, r0, m0, '$SYS/broker/connection/client/bar/state', ... (1 bytes))
     1772518039: Sending PUBLISH to foo (d0, q0, r0, m0, '$SYS/broker/connection/client/bar/state', ... (1 bytes))
     1772518039: Sending PUBLISH to foo (d0, q0, r0, m0, '$SYS/broker/connection/client/bar/state', ... (1 bytes))
     

6. run a third MQTT client (it can be whatever, let's just use mosquitto_sub again for the sake of the test) on a port with the plugin loaded: mosquitto_sub -t '#' -i bar -p 1884
   • notice that there are four events reported by the first subscriber, one per plugin loaded, although this third client connects on a listener that has loaded the plugin only once:

     $SYS/broker/connection/client/bar/state 1
     $SYS/broker/connection/client/bar/state 1
     $SYS/broker/connection/client/bar/state 1
     $SYS/broker/connection/client/bar/state 1
     

   • notice also that the broker reports four PUBLISH:

     1772518048: New connection from 10.0.2.100:58406 on port 1883.
     1772518048: New client connected from 10.0.2.100:58406 as bar (p4, c1, k60).
     1772518048: No will message specified.
     1772518048: Sending CONNACK to bar (0, 0)
     1772518048: Sending PUBLISH to foo (d0, q0, r0, m0, '$SYS/broker/connection/client/bar/state', ... (1 bytes))
     1772518048: Sending PUBLISH to foo (d0, q0, r0, m0, '$SYS/broker/connection/client/bar/state', ... (1 bytes))
     1772518048: Sending PUBLISH to foo (d0, q0, r0, m0, '$SYS/broker/connection/client/bar/state', ... (1 bytes))
     1772518048: Sending PUBLISH to foo (d0, q0, r0, m0, '$SYS/broker/connection/client/bar/state', ... (1 bytes))
     

7. terminate that third client with Ctrl-C:
   • notice that there are four events reported by the first subscriber, one per plugin loaded:

     $SYS/broker/connection/client/bar/state 0
     $SYS/broker/connection/client/bar/state 0
     $SYS/broker/connection/client/bar/state 0
     $SYS/broker/connection/client/bar/state 0
     

   • notice also that the broker reports four PUBLISH:

     1772518052: Received DISCONNECT from bar
     1772518052: Client bar [10.0.2.100:58406] disconnected.
     1772518052: Sending PUBLISH to foo (d0, q0, r0, m0, '$SYS/broker/connection/client/bar/state', ... (1 bytes))
     1772518052: Sending PUBLISH to foo (d0, q0, r0, m0, '$SYS/broker/connection/client/bar/state', ... (1 bytes))
     1772518052: Sending PUBLISH to foo (d0, q0, r0, m0, '$SYS/broker/connection/client/bar/state', ... (1 bytes))
     1772518052: Sending PUBLISH to foo (d0, q0, r0, m0, '$SYS/broker/connection/client/bar/state', ... (1 bytes))
     

---

Expected behaviour:

Only the EVT_CONNECT callbacks from the plugins loaded by the listener the client connects to, should be called, like all other callbacks behave.

Notice that EVT_DISCONNECT callbacks have the same issue as for EVT_CONNECT.

[ymorin-orange]

By applying the patch below on the master branch as of now (commit 3a76714):

diff --git a/src/plugin_connect.c b/src/plugin_connect.c
index f0b706ec..947b4945 100644
--- a/src/plugin_connect.c
+++ b/src/plugin_connect.c
@@ -39,10 +39,12 @@ static void plugin__handle_connect_single(struct mosquitto__security_options *op
 void plugin__handle_connect(struct mosquitto *context)
 {
 	/* Global plugins */
+	mosquitto_log_printf(MOSQ_LOG_DEBUG, "mosquitto: connect event, global plugins");
 	plugin__handle_connect_single(&db.config->security_options, context);
 
 	/* Per listener plugins */
 	if(context->listener){
+		mosquitto_log_printf(MOSQ_LOG_DEBUG, "mosquitto: connect event, per listener plugins");
 		plugin__handle_connect_single(context->listener->security_options, context);
 	}
 }

we can see that the connect callbacks are called as if they were part of the "global plugins":

1772622630: mosquitto: connect event, global plugins
1772622630: connect: for 'foo123' 'foobar' on plugin 0x714cc4b89590
1772622630: connect: for 'foo123' 'foobar' on plugin 0x714cc4b894d0
1772622630: mosquitto: connect event, per listener plugins

The test plugin "just" registers an EVT_CONNECT callback that prints a trace as seen above:

struct mosquitto_evt_connect *ed = (struct mosquitto_evt_connect *)event_data;
mosquitto_log_printf(
    MOSQ_LOG_DEBUG,
    "connect: for '%s' '%s' on plugin %p",
    mosquitto_client_username(ed->client),
    mosquitto_client_id(ed->client),
    userdata
)

[ymorin-orange]

Ah, now I realise that I did remove per_listener_settings true, because it is supposed to now be deprecated, and mosquitto warns about it at startup...

But it turns out that, even though there is the new global_plugin option, the plugin option is still global unless per_listener_settings is set to true...

So, what is really the status for per_listener_settings? It is marked deprecated and that it will be removed with version 3.0, but for some setups, it is still required. Quid?