submariner-lighthouse-coredns can't bind port 53 #2001
Description
What happened:
My submariner-lighthouse-coredns can't bind port 53,here is the log :
submariner-lighthouse-coredns version: release-0.20-20be14112071
maxprocs: Leaving GOMAXPROCS=96: CPU quota undefined
W1226 03:21:06.583084 1 client_config.go:667] Neither --kubeconfig nor --master was specified. Using the inClusterConfig. This might not work.
2025-12-26T03:21:06.583Z INF ..ateway/controller.go:65 Gateway Setting localClusterID from env: "kubernetes"
2025-12-26T03:21:06.597Z INF ..ateway/controller.go:85 Gateway Starting Gateway status Controller
2025-12-26T03:21:06.698Z INF ..solver/controller.go:52 Resolver Starting Resolver Controller
Listen: listen tcp :53: bind: permission denied
I find that in the submariner-lighthouse-coredns deployment yaml, there is :
securityContext:
allowPrivilegeEscalation: false
capabilities:
add:
- net_bind_service
drop:
- all
If I change net_bind_service to NET_BIND_SERVICE, the pod is running, and the log is:
submariner-lighthouse-coredns version: release-0.20-20be14112071
maxprocs: Leaving GOMAXPROCS=96: CPU quota undefined
W1226 03:23:43.354375 1 client_config.go:667] Neither --kubeconfig nor --master was specified. Using the inClusterConfig. This might not work.
2025-12-26T03:23:43.354Z INF ..ateway/controller.go:65 Gateway Setting localClusterID from env: "kubernetes"
2025-12-26T03:23:43.368Z INF ..ateway/controller.go:85 Gateway Starting Gateway status Controller
2025-12-26T03:23:43.469Z INF ..solver/controller.go:52 Resolver Starting Resolver Controller
clusterset.local.:53
CoreDNS-1.12.0
linux/arm64, go1.23.8,
If I have a way to permanently change this configuration parameter? Or there is another way? Please tell me. Thanks.
What you expected to happen:
How to reproduce it (as minimally and precisely as possible):
Anything else we need to know?:
Environment:
- Diagnose information (use
subctl diagnose all):
✓ Checking Submariner support for the Kubernetes version
✓ Kubernetes version "v1.27.1" is supported
✓ Globalnet deployment detected - checking that globalnet CIDRs do not overlap
✓ Checking DaemonSet "submariner-gateway"
✓ Checking DaemonSet "submariner-routeagent"
✓ Checking DaemonSet "submariner-globalnet"
✓ Checking DaemonSet "submariner-metrics-proxy"
✓ Checking Deployment "submariner-lighthouse-agent"
✗ Checking Deployment "submariner-lighthouse-coredns"
✗ The desired number of replicas for Deployment "submariner-lighthouse-coredns" (2) does not match the actual number running (1)
✗ Checking the status of all Submariner pods
⚠ Pod "submariner-gateway-97lc2" has restarted 1715 times
⚠ Pod "submariner-globalnet-j6vgv" has restarted 1616 times
⚠ Pod "submariner-lighthouse-coredns-7f678fb5f7-b9rxb" has restarted 8 times
⚠ Pod "submariner-lighthouse-coredns-7f678fb5f7-qn59p" has restarted 8 times
✗ Pod "submariner-routeagent-7lhn4" is not running. (current state is Pending)
✗ Pod "submariner-routeagent-mbfdm" is not running. (current state is Pending)
⚠ Expected one Gateway pod to be labeled as active. Found 0 on nodes []
✗ Checking Submariner support for the CNI network plugin
✗ The detected CNI plugin ("flannel") is not supported by Submariner. Supported plugins: [generic canal-flannel weave-net OpenShiftSDN OVNKubernetes calico kindnet]
✗ Checking gateway connections
✗ No gateways were detected
✓ Checking route agent connections
✓ There are no remote endpoint connections on route agent "master1"
✓ Checking Submariner support for the kube-proxy mode
✓ The kube-proxy mode is supported
✗ Checking that firewall configuration allows intra-cluster VXLAN traffic
✗ Unable to obtain a remote endpoint: endpoints.submariner.io "remote Endpoint" not found
✗ Checking that Globalnet is correctly configured and functioning
✗ Found 0 ClusterGlobalEgressIP resources but only the default instance ("cluster-egress.submariner.io") is supported
✗ Couldn't find the default ClusterGlobalEgressIP resource("cluster-egress.submariner.io")
- Gather information (use
subctl gather):
Cluster "kubernetes"
Gathering information from cluster "kubernetes"
✗ Gathering connectivity logs
✓ Found 1 pods matching label selector "app=submariner-gateway"
⚠ Found logs for previous instances of pod "submariner-gateway-97lc2"
✓ Found 3 pods matching label selector "app=submariner-routeagent"
✗ Error outputting current log for pod "submariner-routeagent-7lhn4": error opening log stream: the server has asked for the client to provide credentials ( pods/log submariner-routeagent-7lhn4)
✗ Error outputting current log for pod "submariner-routeagent-7lhn4": error opening log stream: the server has asked for the client to provide credentials ( pods/log submariner-routeagent-7lhn4)
✗ Error outputting current log for pod "submariner-routeagent-mbfdm": error opening log stream: Get "https://141.146.76.81:10250/containerLogs/submariner-operator/submariner-routeagent-mbfdm/submariner-routeagent-init": dial tcp 141.146.76.81:10250: connect: connection refused
✗ Error outputting current log for pod "submariner-routeagent-mbfdm": error opening log stream: Get "https://141.146.76.81:10250/containerLogs/submariner-operator/submariner-routeagent-mbfdm/submariner-routeagent": dial tcp 141.146.76.81:10250: connect: connection refused
✓ Found 1 pods matching label selector "app=submariner-metrics-proxy"
✓ Found 1 pods matching label selector "app=submariner-globalnet"
⚠ Found logs for previous instances of pod "submariner-globalnet-j6vgv"
✓ Found 0 pods matching label selector "app=submariner-addon"
✗ Gathering connectivity resources
✓ Gathering CNI data from 3 pods matching label selector "app=submariner-routeagent"
✗ Error running "ip -d a" on pod "submariner-routeagent-7lhn4": unable to upgrade connection: Unauthorized
✗ Error running "ip -d l" on pod "submariner-routeagent-7lhn4": unable to upgrade connection: Unauthorized
✗ Error running "ip route show" on pod "submariner-routeagent-7lhn4": unable to upgrade connection: Unauthorized
✗ Error running "ip rule list" on pod "submariner-routeagent-7lhn4": unable to upgrade connection: Unauthorized
✗ Error running "ip rule show table 150" on pod "submariner-routeagent-7lhn4": unable to upgrade connection: Unauthorized
✗ Error running "sysctl -a" on pod "submariner-routeagent-7lhn4": unable to upgrade connection: Unauthorized
✗ Error running "ipset list" on pod "submariner-routeagent-7lhn4": unable to upgrade connection: Unauthorized
✗ Error running "iptables -L -n -v --line-numbers" on pod "submariner-routeagent-7lhn4": unable to upgrade connection: Unauthorized
✗ Error running "iptables -L -n -v --line-numbers -t nat" on pod "submariner-routeagent-7lhn4": unable to upgrade connection: Unauthorized
✗ Error running "iptables -L -n -v --line-numbers -t mangle" on pod "submariner-routeagent-7lhn4": unable to upgrade connection: Unauthorized
✗ Error running "iptables-save -c" on pod "submariner-routeagent-7lhn4": unable to upgrade connection: Unauthorized
✗ Error running "ipset list" on pod "submariner-routeagent-bhtfd": unable to upgrade connection: error dialing backend: unknown scheme: ws
✗ Error running "ip -d a" on pod "submariner-routeagent-bhtfd": unable to upgrade connection: error dialing backend: unknown scheme: ws
✗ Error running "ip -d l" on pod "submariner-routeagent-bhtfd": unable to upgrade connection: error dialing backend: unknown scheme: ws
✗ Error running "ip route show" on pod "submariner-routeagent-bhtfd": unable to upgrade connection: error dialing backend: unknown scheme: ws
✗ Error running "ip rule list" on pod "submariner-routeagent-bhtfd": unable to upgrade connection: error dialing backend: unknown scheme: ws
✗ Error running "ip rule show table 150" on pod "submariner-routeagent-bhtfd": unable to upgrade connection: error dialing backend: unknown scheme: ws
✗ Error running "sysctl -a" on pod "submariner-routeagent-bhtfd": unable to upgrade connection: error dialing backend: unknown scheme: ws
✗ Error running "iptables -L -n -v --line-numbers -t nat" on pod "submariner-routeagent-bhtfd": unable to upgrade connection: error dialing backend: unknown scheme: ws
✗ Error running "iptables -L -n -v --line-numbers -t mangle" on pod "submariner-routeagent-bhtfd": unable to upgrade connection: error dialing backend: unknown scheme: ws
✗ Error running "iptables-save -c" on pod "submariner-routeagent-bhtfd": unable to upgrade connection: error dialing backend: unknown scheme: ws
✗ Error running "iptables -L -n -v --line-numbers" on pod "submariner-routeagent-bhtfd": unable to upgrade connection: error dialing backend: unknown scheme: ws
✗ Error running "sysctl -a" on pod "submariner-routeagent-mbfdm": error dialing backend: dial tcp 141.146.76.81:10250: connect: connection refused
✗ Error running "ipset list" on pod "submariner-routeagent-mbfdm": error dialing backend: dial tcp 141.146.76.81:10250: connect: connection refused
✗ Error running "ip -d a" on pod "submariner-routeagent-mbfdm": error dialing backend: dial tcp 141.146.76.81:10250: connect: connection refused
✗ Error running "ip -d l" on pod "submariner-routeagent-mbfdm": error dialing backend: dial tcp 141.146.76.81:10250: connect: connection refused
✗ Error running "ip route show" on pod "submariner-routeagent-mbfdm": error dialing backend: dial tcp 141.146.76.81:10250: connect: connection refused
✗ Error running "ip rule list" on pod "submariner-routeagent-mbfdm": error dialing backend: dial tcp 141.146.76.81:10250: connect: connection refused
✗ Error running "ip rule show table 150" on pod "submariner-routeagent-mbfdm": error dialing backend: dial tcp 141.146.76.81:10250: connect: connection refused
✗ Error running "iptables -L -n -v --line-numbers" on pod "submariner-routeagent-mbfdm": error dialing backend: dial tcp 141.146.76.81:10250: connect: connection refused
✗ Error running "iptables -L -n -v --line-numbers -t nat" on pod "submariner-routeagent-mbfdm": error dialing backend: dial tcp 141.146.76.81:10250: connect: connection refused
✗ Error running "iptables -L -n -v --line-numbers -t mangle" on pod "submariner-routeagent-mbfdm": error dialing backend: dial tcp 141.146.76.81:10250: connect: connection refused
✗ Error running "iptables-save -c" on pod "submariner-routeagent-mbfdm": error dialing backend: dial tcp 141.146.76.81:10250: connect: connection refused
✓ Gathering CNI data from 1 pods matching label selector "app=submariner-gateway"
✓ Gathering cable driver data from 1 pods matching label selector "app=submariner-gateway"
✓ Found 0 endpoints in namespace "submariner-operator"
✓ Found 0 clusters in namespace "submariner-operator"
✓ Found 0 gateways in namespace "submariner-operator"
✓ Found 1 routeagents in namespace "submariner-operator"
✓ Found 0 clusterglobalegressips in namespace ""
✓ Found 0 globalegressips in namespace ""
✓ Found 0 globalingressips in namespace ""
⚠ Gathering service-discovery logs
✓ Found 4 pods matching label selector "component=submariner-lighthouse"
⚠ Found logs for previous instances of pod "submariner-lighthouse-coredns-7f678fb5f7-b9rxb"
⚠ Found logs for previous instances of pod "submariner-lighthouse-coredns-7f678fb5f7-qn59p"
✓ Found 2 pods matching label selector "k8s-app=kube-dns"
⚠ Found logs for previous instances of pod "coredns-5d78c9869d-8gxlk"
⚠ Found logs for previous instances of pod "coredns-5d78c9869d-z76vc"
✓ Gathering service-discovery resources
✓ Found 0 serviceexports in namespace ""
✓ Found 0 serviceimports in namespace ""
✓ Found 0 endpointslices by label selector "endpointslice.kubernetes.io/managed-by=lighthouse-agent.submariner.io" in namespace ""
✓ Found 1 configmaps by label selector "component=submariner-lighthouse" in namespace "submariner-operator"
✓ Found 1 configmaps by field selector "metadata.name=coredns" in namespace "kube-system"
✓ Found 0 services by label selector "submariner.io/exportedServiceRef" in namespace ""
✓ Gathering broker logs
✓ Gathering broker resources
✓ Found 0 endpoints in namespace "submariner-k8s-broker"
✓ Found 0 clusters in namespace "submariner-k8s-broker"
✓ Found 0 endpointslices by label selector "endpointslice.kubernetes.io/managed-by=lighthouse-agent.submariner.io" in namespace "submariner-k8s-broker"
✓ Found 0 serviceimports in namespace "submariner-k8s-broker"
✓ Gathering operator logs
✓ Found 1 pods matching label selector "name=submariner-operator"
✓ Gathering operator resources
✓ Found 1 submariners in namespace "submariner-operator"
✓ Found 1 servicediscoveries in namespace "submariner-operator"
✓ Found 1 deployments by field selector "metadata.name=submariner-operator" in namespace "submariner-operator"
✓ Found 0 pods by field selector "metadata.name=submariner-operator" in namespace "submariner-operator"
✓ Found 1 daemonsets by label selector "app=submariner-gateway" in namespace "submariner-operator"
✓ Found 1 pods by label selector "app=submariner-gateway" in namespace "submariner-operator"
✓ Found 1 daemonsets by label selector "app=submariner-metrics-proxy" in namespace "submariner-operator"
✓ Found 1 pods by label selector "app=submariner-metrics-proxy" in namespace "submariner-operator"
✓ Found 1 daemonsets by label selector "app=submariner-routeagent" in namespace "submariner-operator"
✓ Found 3 pods by label selector "app=submariner-routeagent" in namespace "submariner-operator"
✓ Found 1 daemonsets by label selector "app=submariner-globalnet" in namespace "submariner-operator"
✓ Found 1 pods by label selector "app=submariner-globalnet" in namespace "submariner-operator"
✓ Found 1 deployments by label selector "app=submariner-lighthouse-agent" in namespace "submariner-operator"
✓ Found 1 pods by label selector "app=submariner-lighthouse-agent" in namespace "submariner-operator"
✓ Found 1 deployments by label selector "app=submariner-lighthouse-coredns" in namespace "submariner-operator"
✓ Found 3 pods by label selector "app=submariner-lighthouse-coredns" in namespace "submariner-operator"
✓ Found 0 services by field selector "metadata.name=submariner-gateway" in namespace "submariner-operator"
Files are stored under directory "submariner-20251226032746/kubernetes"
subctl version: devel
- Cloud provider or hardware configuration:
- Install tools:
subctl - Others: