netbird v0.24.3 vs 0.24.4, breaking flannel

[siredmar]

I run a kubernetes cluster that uses flannel as CNI. I have a dummy interface called edge0 and some iptables rules that forward incoming/outgoing to/from the netbird interface wt0.
I narrowed it down to version 0.24.3 that works. Any later version breaks behavior and flannel is not able to connect to the other peers even though the pings to the other peers work just fine.

So my question is: can a project maintainer tell me what changes have been made that may break things between 0.24.3 and 0.24.4?
Is there a way (maybe undocumented flag or env) that can be used to let current netbird releases behave like 0.24.3?

[pappz]

Hello @siredmar
Probably this change cause your issue. In this release we implemented a really important firewall modification. Because of it has some required refactor.
Could you me example iptables rules what cause your issue after the NetBird agent start?

[siredmar]

Hi @pappz
thanks for responding to my issue!

Here is some information for you to understand the context and the use case.

We are talking about a small embedded Linux device. We are running this device as a kubernetes node. For a CNI plugin like flannel to start up properly there must be a constant interface up and running. So for the workload on the device to run properly even if the device reboots and keeps being offline, there must be some interface that meets flannels requirements.

So, on boot-up an dummy interface called edge0 is created using this script.

#!/bin/bash

# Name of the dummy interface
dummy_interface="edge0"
mac_address_file="/etc/edge0.mac"
ip_address="192.168.168.1"
# Create or read MAC address from the file
if [ -f "$mac_address_file" ]; then
  mac_address=$(cat "$mac_address_file")
else
  mac_address=$(echo $FQDN|md5sum|sed 's/^\(..\)\(..\)\(..\)\(..\)\(..\).*$/02:\1:\2:\3:\4:\5/')
  echo "$mac_address" > "$mac_address_file"
fi

# Create the dummy interface and assign the IP address and subnet
ip link add name edge0 type dummy
ip link set dev edge0 address "$mac_address"
ip addr add "$ip_address/24" dev edge0
ip link set dev edge0 up

echo 1 > /proc/sys/net/ipv4/ip_forward
exit 0

The script creates edge0 with an initially randomized but persisted MAC address beyond reboots. It sets the fixed IP address of 192.168.168.1 to this interface.

After connecting to netbird and wt0 is created this script is ran using some udev rules

#!/bin/bash
ip_file="/usr/local/etc/wt0.ip"
wt0_info=$(ip addr show dev wt0 | grep -oE 'inet [0-9.]+' | awk '{print $2}')
echo $wt0_info > $ip_file

iptables -A FORWARD -i wt0 -o edge0 -p tcp --dport 1:65535 -j ACCEPT
iptables -A FORWARD -i edge0 -o wt0 -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
iptables -t nat -A PREROUTING -i wt0 -p tcp --dport 1:65535 -j DNAT --to-destination 192.168.168.1
iptables -t nat -A POSTROUTING -o wt0 -j MASQUERADE

The script reads the IP address from wt0 and stores it in a file for flannel to mount it and use it as the public-ip argument. Flannel also interacts with edge0 (with parameter --iface)
Here you find the iptables rules that redirects all incoming and outgoing traffic from and to wt0/edge0.
When the kubelet is startet it also uses 192.168.168.1 binding edge0. This means that both flannel and kubelet uses the VPN.

These are the only firewall rules (kube-proxy excluded) we set and like i said using netbird 0.24.3 worked like a charm.

[pappz]

Thank you for the detailed explanation. The key difference in this version is that in the older version the agent operated on the input and output chains. After this version we extended it to the routed traffic also.
Maybe if you use insert instead of append it could solve your problem.

[siredmar]

I tried

iptables -A FORWARD -i wt0 -o edge0 -p tcp --dport 1:65535 -j ACCEPT
iptables -A FORWARD -i edge0 -o wt0 -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
iptables -t nat -A PREROUTING -i wt0 -p tcp --dport 1:65535 -j DNAT --to-destination 192.168.168.1
iptables -t nat -A POSTROUTING -o wt0 -j MASQUERADE

But the same behavior. Flannel is not able to communicate using my edge0 interface.

[pappz]

The NetBird agent can support Nftables and iptabales. I am not sure in that in your case what is the preferred but could you send me the output of this command:

iptables -L -n

[siredmar]

sure. I'm not an iptables expert. Here's the output

# iptables -L -n
Chain INPUT (policy ACCEPT)
target     prot opt source               destination         
KUBE-FIREWALL  all  --  0.0.0.0/0            0.0.0.0/0           

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination         
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0            tcp dpts:1:65535
DOCKER-USER  all  --  0.0.0.0/0            0.0.0.0/0           
DOCKER-ISOLATION-STAGE-1  all  --  0.0.0.0/0            0.0.0.0/0           
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
DOCKER     all  --  0.0.0.0/0            0.0.0.0/0           
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0           
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0           
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0            tcp dpts:1:65535
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination         
KUBE-FIREWALL  all  --  0.0.0.0/0            0.0.0.0/0           

Chain DOCKER (1 references)
target     prot opt source               destination         

Chain DOCKER-ISOLATION-STAGE-1 (1 references)
target     prot opt source               destination         
DOCKER-ISOLATION-STAGE-2  all  --  0.0.0.0/0            0.0.0.0/0           
RETURN     all  --  0.0.0.0/0            0.0.0.0/0           

Chain DOCKER-ISOLATION-STAGE-2 (1 references)
target     prot opt source               destination         
DROP       all  --  0.0.0.0/0            0.0.0.0/0           
RETURN     all  --  0.0.0.0/0            0.0.0.0/0           

Chain DOCKER-USER (1 references)
target     prot opt source               destination         
RETURN     all  --  0.0.0.0/0            0.0.0.0/0           

Chain KUBE-FIREWALL (2 references)
target     prot opt source               destination         
DROP       all  --  0.0.0.0/0            0.0.0.0/0            /* kubernetes firewall for dropping marked packets */ mark match 0x8000/0x8000
DROP       all  -- !127.0.0.0/8          127.0.0.0/8          /* block incoming localnet connections */ ! ctstate RELATED,ESTABLISHED,DNAT

Chain KUBE-KUBELET-CANARY (0 references)
target     prot opt source               destination         

Chain KUBE-PROXY-CANARY (0 references)
target     prot opt source               destination 

[siredmar]

Here is also the log if nft show ruleset running netbird 0.25.5

table ip netbird {
	set nb0000001 {
		type ipv4_addr
		flags dynamic
		elements = { 0.0.0.0 }
	}

	set nb0000002 {
		type ipv4_addr
		flags dynamic
		elements = { 0.0.0.0 }
	}

	chain netbird-rt-fwd {
	}

	chain netbird-rt-nat {
		type nat hook postrouting priority srcnat - 1; policy accept;
	}

	chain netbird-acl-input-rules {
		iifname "wt0" accept
	}

	chain netbird-acl-output-rules {
		oifname "wt0" accept
	}

	chain netbird-acl-input-filter {
		type filter hook input priority filter; policy accept;
		iifname "wt0" ip saddr 100.127.0.0/16 ip daddr != 100.127.0.0/16 accept
		iifname "wt0" ip saddr != 100.127.0.0/16 ip daddr 100.127.0.0/16 accept
		iifname "wt0" ip saddr 100.127.0.0/16 ip daddr 100.127.0.0/16 jump netbird-acl-input-rules
		iifname "wt0" drop
	}

	chain netbird-acl-output-filter {
		type filter hook output priority filter; policy accept;
		oifname "wt0" ip saddr != 100.127.0.0/16 ip daddr 100.127.0.0/16 accept
		oifname "wt0" ip saddr 100.127.0.0/16 ip daddr != 100.127.0.0/16 accept
		oifname "wt0" ip saddr 100.127.0.0/16 ip daddr 100.127.0.0/16 jump netbird-acl-output-rules
		oifname "wt0" drop
	}

	chain netbird-acl-forward-filter {
		type filter hook forward priority filter; policy accept;
		iifname "wt0" jump netbird-rt-fwd
		oifname "wt0" jump netbird-rt-fwd
		iifname "wt0" meta mark 0x000007e4 accept
		oifname "wt0" meta mark 0x000007e4 accept
		iifname "wt0" jump netbird-acl-input-rules
		iifname "wt0" drop
	}

	chain netbird-acl-prerouting-filter {
		type filter hook prerouting priority mangle; policy accept;
		iifname "wt0" ip saddr != 100.127.0.0/16 ip daddr 100.127.181.129 meta mark set 0x000007e4
	}
}

See details for full rules output

# nft list ruleset
table ip filter {
	chain DOCKER {
	}

	chain DOCKER-ISOLATION-STAGE-1 {
		iifname "docker0" oifname != "docker0" counter packets 0 bytes 0 jump DOCKER-ISOLATION-STAGE-2
		counter packets 0 bytes 0 return
	}

	chain DOCKER-ISOLATION-STAGE-2 {
		oifname "docker0" counter packets 0 bytes 0 drop
		counter packets 0 bytes 0 return
	}

	chain FORWARD {
		type filter hook forward priority filter; policy accept;
		iifname "edge0" oifname "wt0" ct state related,established counter packets 0 bytes 0 accept
		iifname "wt0" oifname "edge0" meta l4proto tcp tcp dport 1-65535 counter packets 0 bytes 0 accept
		counter packets 0 bytes 0 jump DOCKER-USER
		counter packets 0 bytes 0 jump DOCKER-ISOLATION-STAGE-1
		oifname "docker0" ct state related,established counter packets 0 bytes 0 accept
		oifname "docker0" counter packets 0 bytes 0 jump DOCKER
		iifname "docker0" oifname != "docker0" counter packets 0 bytes 0 accept
		iifname "docker0" oifname "docker0" counter packets 0 bytes 0 accept
		iifname "wt0" oifname "edge0" meta l4proto tcp tcp dport 1-65535 counter packets 0 bytes 0 accept
		iifname "edge0" oifname "wt0" ct state related,established counter packets 0 bytes 0 accept
	}

	chain DOCKER-USER {
		counter packets 0 bytes 0 return
	}

	chain KUBE-FIREWALL {
		 mark and 0x8000 == 0x8000 counter packets 0 bytes 0 drop
		ip saddr != 127.0.0.0/8 ip daddr 127.0.0.0/8  ct status dnat counter packets 0 bytes 0 drop
	}

	chain OUTPUT {
		type filter hook output priority filter; policy accept;
		counter packets 1649176 bytes 161730220 jump KUBE-FIREWALL
	}

	chain INPUT {
		type filter hook input priority filter; policy accept;
		counter packets 1649952 bytes 240227628 jump KUBE-FIREWALL
	}

	chain KUBE-KUBELET-CANARY {
	}

	chain KUBE-PROXY-CANARY {
	}
}
table ip nat {
	chain DOCKER {
		iifname "docker0" counter packets 0 bytes 0 return
	}

	chain POSTROUTING {
		type nat hook postrouting priority srcnat; policy accept;
		oifname "wt0" counter packets 66961 bytes 4017820 masquerade 
		 counter packets 344083 bytes 30106893 jump KUBE-POSTROUTING
		oifname != "docker0" ip saddr 172.17.0.0/16 counter packets 0 bytes 0 masquerade 
		oifname "wt0" counter packets 0 bytes 0 masquerade 
	}

	chain PREROUTING {
		type nat hook prerouting priority dstnat; policy accept;
		iifname "wt0" meta l4proto tcp tcp dport 1-65535 counter packets 3 bytes 4412 dnat to 192.168.168.1
		fib daddr type local counter packets 36 bytes 4792 jump DOCKER
		iifname "wt0" meta l4proto tcp tcp dport 1-65535 counter packets 0 bytes 0 dnat to 192.168.168.1
	}

	chain OUTPUT {
		type nat hook output priority -100; policy accept;
		ip daddr != 127.0.0.0/8 fib daddr type local counter packets 67281 bytes 5511149 jump DOCKER
	}

	chain KUBE-MARK-DROP {
		counter packets 0 bytes 0 meta mark set mark or 0x8000 
	}

	chain KUBE-MARK-MASQ {
		counter packets 0 bytes 0 meta mark set mark or 0x4000 
	}

	chain KUBE-POSTROUTING {
		mark and 0x4000 != 0x4000 counter packets 344083 bytes 30106893 return
		counter packets 0 bytes 0 meta mark set mark xor 0x4000 
		 counter packets 0 bytes 0 masquerade 
	}

	chain KUBE-KUBELET-CANARY {
	}

	chain INPUT {
		type nat hook input priority 100; policy accept;
	}

	chain KUBE-PROXY-CANARY {
	}
}
table ip6 filter {
	chain KUBE-FIREWALL {
		 mark and 0x8000 == 0x8000 counter packets 0 bytes 0 drop
	}

	chain OUTPUT {
		type filter hook output priority filter; policy accept;
		counter packets 14303 bytes 1371054 jump KUBE-FIREWALL
	}

	chain INPUT {
		type filter hook input priority filter; policy accept;
		counter packets 14223 bytes 1933030 jump KUBE-FIREWALL
	}

	chain KUBE-KUBELET-CANARY {
	}

	chain FORWARD {
		type filter hook forward priority filter; policy accept;
	}

	chain KUBE-PROXY-CANARY {
	}
}
table ip6 nat {
	chain KUBE-MARK-DROP {
		counter packets 0 bytes 0 meta mark set mark or 0x8000 
	}

	chain KUBE-MARK-MASQ {
		counter packets 0 bytes 0 meta mark set mark or 0x4000 
	}

	chain KUBE-POSTROUTING {
		mark and 0x4000 != 0x4000 counter packets 13196 bytes 1292738 return
		counter packets 0 bytes 0 meta mark set mark xor 0x4000 
		 counter packets 0 bytes 0 masquerade  random-fully 
	}

	chain POSTROUTING {
		type nat hook postrouting priority srcnat; policy accept;
		 counter packets 13196 bytes 1292738 jump KUBE-POSTROUTING
	}

	chain KUBE-KUBELET-CANARY {
	}

	chain PREROUTING {
		type nat hook prerouting priority dstnat; policy accept;
	}

	chain INPUT {
		type nat hook input priority 100; policy accept;
	}

	chain OUTPUT {
		type nat hook output priority -100; policy accept;
	}

	chain KUBE-PROXY-CANARY {
	}
}
table ip netbird {
	set nb0000001 {
		type ipv4_addr
		flags dynamic
		elements = { 0.0.0.0 }
	}

	set nb0000002 {
		type ipv4_addr
		flags dynamic
		elements = { 0.0.0.0 }
	}

	chain netbird-rt-fwd {
	}

	chain netbird-rt-nat {
		type nat hook postrouting priority srcnat - 1; policy accept;
	}

	chain netbird-acl-input-rules {
		iifname "wt0" accept
	}

	chain netbird-acl-output-rules {
		oifname "wt0" accept
	}

	chain netbird-acl-input-filter {
		type filter hook input priority filter; policy accept;
		iifname "wt0" ip saddr 100.127.0.0/16 ip daddr != 100.127.0.0/16 accept
		iifname "wt0" ip saddr != 100.127.0.0/16 ip daddr 100.127.0.0/16 accept
		iifname "wt0" ip saddr 100.127.0.0/16 ip daddr 100.127.0.0/16 jump netbird-acl-input-rules
		iifname "wt0" drop
	}

	chain netbird-acl-output-filter {
		type filter hook output priority filter; policy accept;
		oifname "wt0" ip saddr != 100.127.0.0/16 ip daddr 100.127.0.0/16 accept
		oifname "wt0" ip saddr 100.127.0.0/16 ip daddr != 100.127.0.0/16 accept
		oifname "wt0" ip saddr 100.127.0.0/16 ip daddr 100.127.0.0/16 jump netbird-acl-output-rules
		oifname "wt0" drop
	}

	chain netbird-acl-forward-filter {
		type filter hook forward priority filter; policy accept;
		iifname "wt0" jump netbird-rt-fwd
		oifname "wt0" jump netbird-rt-fwd
		iifname "wt0" meta mark 0x000007e4 accept
		oifname "wt0" meta mark 0x000007e4 accept
		iifname "wt0" jump netbird-acl-input-rules
		iifname "wt0" drop
	}

	chain netbird-acl-prerouting-filter {
		type filter hook prerouting priority mangle; policy accept;
		iifname "wt0" ip saddr != 100.127.0.0/16 ip daddr 100.127.181.129 meta mark set 0x000007e4
	}
}
table ip mangle {
	chain KUBE-KUBELET-CANARY {
	}

	chain PREROUTING {
		type filter hook prerouting priority mangle; policy accept;
	}

	chain INPUT {
		type filter hook input priority mangle; policy accept;
	}

	chain FORWARD {
		type filter hook forward priority mangle; policy accept;
	}

	chain OUTPUT {
		type route hook output priority mangle; policy accept;
	}

	chain POSTROUTING {
		type filter hook postrouting priority mangle; policy accept;
	}

	chain KUBE-PROXY-CANARY {
	}
}
table ip6 mangle {
	chain KUBE-KUBELET-CANARY {
	}

	chain PREROUTING {
		type filter hook prerouting priority mangle; policy accept;
	}

	chain INPUT {
		type filter hook input priority mangle; policy accept;
	}

	chain FORWARD {
		type filter hook forward priority mangle; policy accept;
	}

	chain OUTPUT {
		type route hook output priority mangle; policy accept;
	}

	chain POSTROUTING {
		type filter hook postrouting priority mangle; policy accept;
	}

	chain KUBE-PROXY-CANARY {
	}
}

I can see some netbrid entries

[siredmar]

These are the the nft rules for netbird-acl running netbird 0.24.3

table ip netbird-acl {
	set nb0000001 {
		type ipv4_addr
		flags dynamic
		elements = { 0.0.0.0 }
	}

	set nb0000002 {
		type ipv4_addr
		flags dynamic
		elements = { 0.0.0.0 }
	}

	chain netbird-acl-input-filter {
		type filter hook input priority filter; policy accept;
		iifname "wt0" accept
		iifname "wt0" ip saddr != 100.127.0.0/16 accept
		iifname "wt0" drop
	}

	chain netbird-acl-output-filter {
		type filter hook output priority filter; policy accept;
		oifname "wt0" accept
		oifname "wt0" ip daddr != 100.127.0.0/16 accept
		oifname "wt0" drop
	}
}

See details for full rules output

table ip filter {
	chain DOCKER {
	}

	chain DOCKER-ISOLATION-STAGE-1 {
		iifname "docker0" oifname != "docker0" counter packets 0 bytes 0 jump DOCKER-ISOLATION-STAGE-2
		counter packets 0 bytes 0 return
	}

	chain DOCKER-ISOLATION-STAGE-2 {
		oifname "docker0" counter packets 0 bytes 0 drop
		counter packets 0 bytes 0 return
	}

	chain FORWARD {
		type filter hook forward priority filter; policy accept;
		iifname "edge0" oifname "wt0" ct state related,established counter packets 0 bytes 0 accept
		iifname "wt0" oifname "edge0" meta l4proto tcp tcp dport 1-65535 counter packets 0 bytes 0 accept
		counter packets 0 bytes 0 jump DOCKER-USER
		counter packets 0 bytes 0 jump DOCKER-ISOLATION-STAGE-1
		oifname "docker0" ct state related,established counter packets 0 bytes 0 accept
		oifname "docker0" counter packets 0 bytes 0 jump DOCKER
		iifname "docker0" oifname != "docker0" counter packets 0 bytes 0 accept
		iifname "docker0" oifname "docker0" counter packets 0 bytes 0 accept
		iifname "wt0" oifname "edge0" meta l4proto tcp tcp dport 1-65535 counter packets 0 bytes 0 accept
		iifname "edge0" oifname "wt0" ct state related,established counter packets 0 bytes 0 accept
	}

	chain DOCKER-USER {
		counter packets 0 bytes 0 return
	}

	chain KUBE-FIREWALL {
		 mark and 0x8000 == 0x8000 counter packets 0 bytes 0 drop
		ip saddr != 127.0.0.0/8 ip daddr 127.0.0.0/8  ct status dnat counter packets 0 bytes 0 drop
	}

	chain OUTPUT {
		type filter hook output priority filter; policy accept;
		counter packets 1656482 bytes 162559663 jump KUBE-FIREWALL
	}

	chain INPUT {
		type filter hook input priority filter; policy accept;
		iifname "wt0" accept
		counter packets 1661057 bytes 250781434 jump KUBE-FIREWALL
	}

	chain KUBE-KUBELET-CANARY {
	}

	chain KUBE-PROXY-CANARY {
	}
}
table ip nat {
	chain DOCKER {
		iifname "docker0" counter packets 0 bytes 0 return
	}

	chain POSTROUTING {
		type nat hook postrouting priority srcnat; policy accept;
		oifname "wt0" counter packets 5 bytes 568 masquerade 
		 counter packets 344359 bytes 30128429 jump KUBE-POSTROUTING
		oifname != "docker0" ip saddr 172.17.0.0/16 counter packets 0 bytes 0 masquerade 
		oifname "wt0" counter packets 0 bytes 0 masquerade 
	}

	chain PREROUTING {
		type nat hook prerouting priority dstnat; policy accept;
		iifname "wt0" meta l4proto tcp tcp dport 1-65535 counter packets 0 bytes 0 dnat to 192.168.168.1
		fib daddr type local counter packets 42 bytes 5632 jump DOCKER
		iifname "wt0" meta l4proto tcp tcp dport 1-65535 counter packets 0 bytes 0 dnat to 192.168.168.1
	}

	chain OUTPUT {
		type nat hook output priority -100; policy accept;
		ip daddr != 127.0.0.0/8 fib daddr type local counter packets 67342 bytes 5515013 jump DOCKER
	}

	chain KUBE-MARK-DROP {
		counter packets 0 bytes 0 meta mark set mark or 0x8000 
	}

	chain KUBE-MARK-MASQ {
		counter packets 0 bytes 0 meta mark set mark or 0x4000 
	}

	chain KUBE-POSTROUTING {
		mark and 0x4000 != 0x4000 counter packets 344359 bytes 30128429 return
		counter packets 0 bytes 0 meta mark set mark xor 0x4000 
		 counter packets 0 bytes 0 masquerade 
	}

	chain KUBE-KUBELET-CANARY {
	}

	chain INPUT {
		type nat hook input priority 100; policy accept;
	}

	chain KUBE-PROXY-CANARY {
	}
}
table ip6 filter {
	chain KUBE-FIREWALL {
		 mark and 0x8000 == 0x8000 counter packets 0 bytes 0 drop
	}

	chain OUTPUT {
		type filter hook output priority filter; policy accept;
		counter packets 14393 bytes 1379163 jump KUBE-FIREWALL
	}

	chain INPUT {
		type filter hook input priority filter; policy accept;
		counter packets 14312 bytes 1944534 jump KUBE-FIREWALL
	}

	chain KUBE-KUBELET-CANARY {
	}

	chain FORWARD {
		type filter hook forward priority filter; policy accept;
	}

	chain KUBE-PROXY-CANARY {
	}
}
table ip6 nat {
	chain KUBE-MARK-DROP {
		counter packets 0 bytes 0 meta mark set mark or 0x8000 
	}

	chain KUBE-MARK-MASQ {
		counter packets 0 bytes 0 meta mark set mark or 0x4000 
	}

	chain KUBE-POSTROUTING {
		mark and 0x4000 != 0x4000 counter packets 13277 bytes 1300134 return
		counter packets 0 bytes 0 meta mark set mark xor 0x4000 
		 counter packets 0 bytes 0 masquerade  random-fully 
	}

	chain POSTROUTING {
		type nat hook postrouting priority srcnat; policy accept;
		 counter packets 13277 bytes 1300134 jump KUBE-POSTROUTING
	}

	chain KUBE-KUBELET-CANARY {
	}

	chain PREROUTING {
		type nat hook prerouting priority dstnat; policy accept;
	}

	chain INPUT {
		type nat hook input priority 100; policy accept;
	}

	chain OUTPUT {
		type nat hook output priority -100; policy accept;
	}

	chain KUBE-PROXY-CANARY {
	}
}
table ip mangle {
	chain KUBE-KUBELET-CANARY {
	}

	chain PREROUTING {
		type filter hook prerouting priority mangle; policy accept;
	}

	chain INPUT {
		type filter hook input priority mangle; policy accept;
	}

	chain FORWARD {
		type filter hook forward priority mangle; policy accept;
	}

	chain OUTPUT {
		type route hook output priority mangle; policy accept;
	}

	chain POSTROUTING {
		type filter hook postrouting priority mangle; policy accept;
	}

	chain KUBE-PROXY-CANARY {
	}
}
table ip6 mangle {
	chain KUBE-KUBELET-CANARY {
	}

	chain PREROUTING {
		type filter hook prerouting priority mangle; policy accept;
	}

	chain INPUT {
		type filter hook input priority mangle; policy accept;
	}

	chain FORWARD {
		type filter hook forward priority mangle; policy accept;
	}

	chain OUTPUT {
		type route hook output priority mangle; policy accept;
	}

	chain POSTROUTING {
		type filter hook postrouting priority mangle; policy accept;
	}

	chain KUBE-PROXY-CANARY {
	}
}
table ip netbird-acl {
	set nb0000001 {
		type ipv4_addr
		flags dynamic
		elements = { 0.0.0.0 }
	}

	set nb0000002 {
		type ipv4_addr
		flags dynamic
		elements = { 0.0.0.0 }
	}

	chain netbird-acl-input-filter {
		type filter hook input priority filter; policy accept;
		iifname "wt0" accept
		iifname "wt0" ip saddr != 100.127.0.0/16 accept
		iifname "wt0" drop
	}

	chain netbird-acl-output-filter {
		type filter hook output priority filter; policy accept;
		oifname "wt0" accept
		oifname "wt0" ip daddr != 100.127.0.0/16 accept
		oifname "wt0" drop
	}
}

[siredmar]

@pappz do you have any idea?