[Docs] Document Calico configuration issue without encapsulation #3180
Conversation
|
Hi @riccardotornesello. Thanks for your PR! I am @adamjensenbot.
Make sure this PR appears in the liqo changelog, adding one of the following labels:
|
|
|
||
| When using Calico in **VXLAN encapsulation mode**, you need to ensure that the encapsulation is enabled for all the traffic, not only for cross-subnet traffic. | ||
|
|
||
| This ensures that the GENEVE traffic is not affected by NAT rules that may be applied on the nodes, which would break the encapsulation. |
There was a problem hiding this comment.
Can you explain better where this NAT is performed. Because, in order to avoid Node IP natting when the geneve traffic leaves the gateway, we have this flag networking.fabric.config.fullMasquerade
In general, it would be better to find a solution to avoid changing the calico setup
There was a problem hiding this comment.
Sorry for the delay in responding.
I think mine is a special case, using k3d, but I had a similar problem on bare-metal clusters with kubeadm, so I'll try again there too. I don't know if the fullMasquerade option will solve this problem.
I'll explain the scenario so you can evaluate it too.
NOTE: VXLANCrossSubnet is the configuration found in the example variable files, but without specifying a value for “encapsulation,” the default setting is IPIP.
Scenario
In this example, a Pod on the Consumer (PO1) is trying to ping a pod on the Provider.
The traffic will be checked on the Consumer's nodes.
- Nodes CIDR: 172.19.0.0/16
- Pods CIDR: 10.200.0.0/16
- Provider CIDR: 10.71.0.0/16
Results
Case 1: VXLAN (working)
In this scenario, VXLAN encapsulation is applied, so there is no NAT on the tunnel packet.
Step 1:
cali781879d0173 In IPv4: 10.200.0.1 > 10.71.0.3: ICMP echo request
liqo.79mdz7jl4m Out IPv4: 10.70.0.0 > 10.71.0.3: ICMP echo request
vxlan.calico Out IPv4: 10.200.208.128.42083 > 10.200.0.2.6091: UDP
eth0 Out IPv4: 172.19.0.4.39049 > 172.19.0.3.4789: VXLAN, flags [I] (0x08), vni 4096
Step 2:
eth0 In IPv4: 172.19.0.4.39049 > 172.19.0.3.4789: VXLAN, flags [I] (0x08), vni 4096
vxlan.calico In IPv4: 10.200.208.128.42083 > 10.200.0.2.6091: UDP
calib7d0f79f595 Out IPv4: 10.200.208.128.42083 > 10.200.0.2.6091: UDP
Step 3: out of scope
Step 4:
calib7d0f79f595 In IPv4: 10.200.0.2.42083 > 10.200.208.128.6091: UDP
vxlan.calico Out IPv4: 10.200.0.2.42083 > 10.200.208.128.6091: UDP
eth0 Out IPv4: 172.19.0.3.39049 > 172.19.0.4.4789: VXLAN, flags [I] (0x08), vni 4096
Step 5
eth0 In IPv4: 172.19.0.3.39049 > 172.19.0.4.4789: VXLAN, flags [I] (0x08), vni 4096
vxlan.calico In IPv4: 10.200.0.2.42083 > 10.200.208.128.6091: UDP
liqo.79mdz7jl4m In IPv4: 10.71.0.3 > 10.70.0.0: ICMP echo reply
cali781879d0173 Out IPv4: 10.71.0.3 > 10.200.0.1: ICMP echo reply
Case 2: VXLANCrossSubnet (not working)
In this example the request is stuck in a step before: the ARP request in the GENEVE tunnel.
Step 1:
liqo.bgsq4nrt2b Out ARP: Request who-has 10.80.0.4 tell 10.80.0.2
eth0 Out IPv4: 172.19.0.4.1184 > 10.200.0.2.6091: UDP
Something strange happens here: the packet is sent to eth0 even though the destination subnet is different, but the packet reaches the gateway, which even responds.
The response manages to reach the Pod node but is discarded.
NAT is applied in the gateway node, so the Pod node sees the packet arriving from 172.19.0.3 instead of 10.200.0.2.
|
@cheina97 |
3 participants
Description
This PR aims to add a small section regarding Calico configuration to the page on installing liqo.
The suggestion is not to use VXLANCrossSubnet as it breaks Geneve tunnel connections.
How Has This Been Tested?
A k3d cluster was created with Calico configured first with VXLANCrossSubnet encapsulation and then VXLAN.
In the first case, the connection between the gateway and the other nodes did not work, while in the second case it did.