Description
Hey netmaker people,
I am trying to deploy netmaker (0.17.1), but want to use istio as ingress, because in my case everything uses it. UI and API work fine, but I have issue with the MQTT connections.
When I create a new access key and let a client connect via the netclient join ... command it goes into a loop of
Feb 16 20:23:46 plex networkd-dispatcher[108]: WARNING:Unknown index 2565 seen, reloading interface list
Feb 16 20:23:46 plex systemd-networkd[96]: nm-test: Link UP
Feb 16 20:23:46 plex systemd-networkd[96]: nm-test: Gained carrier
Feb 16 20:23:46 plex netclient[125]: [netclient] 2023-02-16 20:23:46 [common.go-162] InitWireguard(): waiting for interface...
Feb 16 20:23:46 plex netclient[125]: [netclient] 2023-02-16 20:23:46 [common.go-190] InitWireguard(): interface ready - netclient.. ENGAGE
Feb 16 20:23:46 plex netclient[125]: [netclient] 2023-02-16 20:23:46 [mqpublish.go-52] checkin(): checkin with server(s) for all networks
Feb 16 20:23:47 plex netclient[125]: [netclient] 2023-02-16 20:23:47 [localport.go-40] UpdateLocalListenPort(): network: test local port has changed from 0 to 48022
Feb 16 20:24:17 plex netclient[125]: [netclient] 2023-02-16 20:24:17 [mqpublish.go-252] publish(): could not connect to broker at broker.netmaker.erpf.de:8883
Feb 16 20:24:17 plex netclient[125]: [netclient] 2023-02-16 20:24:17 [localport.go-47] UpdateLocalListenPort(): could not publish local port change connection timeout
Feb 16 20:24:47 plex netclient[125]: [netclient] 2023-02-16 20:24:47 [mqpublish.go-252] publish(): could not connect to broker at broker.netmaker.erpf.de:8883
Feb 16 20:24:47 plex netclient[125]: [netclient] 2023-02-16 20:24:47 [mqpublish.go-149] Hello(): Network: test error publishing ping, connection timeout
Feb 16 20:24:47 plex netclient[125]: [netclient] 2023-02-16 20:24:47 [mqpublish.go-150] Hello(): running pull on test to reconnect
Feb 16 20:24:47 plex systemd-networkd[96]: nm-test: Link DOWN
Feb 16 20:24:47 plex systemd-networkd[96]: nm-test: Lost carrier
Feb 16 20:24:47 plex netclient[125]: [netclient] 2023-02-16 20:24:47 [common.go-389] informPortChange(): network: test UDP hole punching enabled for node plex
Feb 16 20:24:49 plex networkd-dispatcher[108]: WARNING:Unknown index 2566 seen, reloading interface list
In this case I had istio listening on port 8883 without tls, but I also tried it on port 443 with tls. After thinking that istio in itself could be the issue I decided to test using a NodePort for the 8883 port instead with
---
apiVersion: v1
kind: Service
metadata:
name: {{ include "netmaker.fullname" . }}-mqtt-external
spec:
type: NodePort
ports:
- name: mqtt2
port: 8883
nodePort: 8883
protocol: TCP
targetPort: mqtt2
selector:
app: {{ include "netmaker.fullname" . }}-mqtt
sessionAffinity: NoneAnd this work only sort of. When I set allow_anonymous true I can connect with a mqtt client from my machine via the dns record broker.netmaker.<masked>.de. Also when I allow_anonymous false I can attempt a connection with a mqtt client from my machine and get an Unauthorized (since I don't have an login credentials). So far so good, I would believe connectivity is possible.
But in all scenarios the actual netclient is not able to connect to the broker, neither with allow_anonymous true or false.
One question would be, as this is NodePort from the service and therefore is no TLS on the connection, is this even supposed to work or do I need TLS anyway?
Any hint or tips would be greatly appreciated... :-)