Queue astarte_vmq_plugin_rpc disappearing from RabbitMQ

[Pavinati]

We found an odd crash on Data Updater Plant. It crashed with a timeout after trying to push a force_disconnection message in the VerneMQ RPC queue.

Aug  7 12:48:27 |ERRO| GenServer {Registry.DataUpdater, {"test", <<110, 174, 96, 120, 180, 191, 67, 214, 138, 198, 238, 112, 135, 193, 218, 234>>}} terminating
** (stop) exited in: GenServer.call(Astarte.RPC.AMQP.Client, {:rpc, <<26, 33, 10, 29, 108, 117, 116, 101, 115, 116, 47, 98, 113, 53, 103, 101, 76, 83, 95, 81, 57, 97, 75, 120, 117, 53, 119, 104, 56, 72, 97, 54, 103, 16, 1>>, "vmq_plugin_rpc"}, 5000)
    ** (EXIT) time out
    (elixir 1.14.5) lib/gen_server.ex:1038: GenServer.call/3
    (astarte_data_updater_plant 1.1.0) lib/astarte_data_updater_plant/rpc/vmq_plugin.ex:63: Astarte.DataUpdaterPlant.RPC.VMQPlugin.disconnect/2
    (astarte_data_updater_plant 1.1.0) lib/astarte_data_updater_plant/data_updater/impl.ex:2152: Astarte.DataUpdaterPlant.DataUpdater.Impl.force_disconnection/2
    (astarte_data_updater_plant 1.1.0) lib/astarte_data_updater_plant/data_updater/impl.ex:2131: Astarte.DataUpdaterPlant.DataUpdater.Impl.ask_clean_session/2
    (astarte_data_updater_plant 1.1.0) lib/astarte_data_updater_plant/data_updater/impl.ex:875: Astarte.DataUpdaterPlant.DataUpdater.Impl.handle_data/6
    (astarte_data_updater_plant 1.1.0) lib/astarte_data_updater_plant/data_updater/server.ex:83: Astarte.DataUpdaterPlant.DataUpdater.Server.handle_cast/2
    (stdlib 4.3.1) gen_server.erl:1123: :gen_server.try_dispatch/4
    (stdlib 4.3.1) gen_server.erl:1200: :gen_server.handle_msg/6

After some debugging we found that the astarte_vmq_plugin_rpc queue was not present in RabbitMQ and even killing Astarte.RPC.AMQP.Server (which is responsible for declaring said queue) did not restore it.
in the end we opted for restarting VerneMQ which solved the issue and the queue was declared as expected.

Environment:

• Astarte v1.1.0
• RabbitMQ 3.8.34 (with 3 replicas)
• Astarte VerneMQ 1.1.0