Fix IPv6 deployment variables - #1709
Conversation
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
metal3-io-bot
added
the
size/L
|
Yet another PR about IPv6. Previously the metal3 CI VMs supported only IPv4, but now this has changed and I have been able to test these with VM having native IPv6 spport. So these changes are based on the assumption that host machine have globally routable IPv6 address, which means that no tricks is needed to make IPv6 provisioning work. With these changes provisioning an IPv6 only cluster should work, meaning that the provisioned cluster has 0 IPv4 addresses (excluding loopback) and the network boot is done over IPv6 and also IPA image has only IPv6 addresses. Pivoting won't still work. This is because CRI-O in the provisioned cluster does not allow IPv6 addresses in the image references. CRI-O uses external image reference parse, which is here: https://github.com/podman-container-tools/container-libs/blob/image/v5.40.0/image/docker/reference/regexp.go The comments there state
So in our case when the name starts with If this PR gets merged, then I will first rebase (so that they can serve as proof of concepts if needed) and then close the two other PRs
These are probably not needed/wanted because it is reasonable to expect that the host machine has globally routable IPv6 address. |
1428294 to
466f7ae
Compare
| before: "name: FELIX_HEALTHENABLED" | ||
| regexp: "false" | ||
| replace: "true" | ||
| when: IP_STACK == "v6" |
There was a problem hiding this comment.
How about dual stack? Also, I don't see CALICO_IPV6POOL_CIDR being set anywhere.
There was a problem hiding this comment.
I have not tested dual stack at all because I have been afraid that I would not notice IPv6 failures if the system falls back to v4. But I think we can configure dual stack to the provisioned cluster without any problems. I will test it.
I did not add CALICO_IPV6POOL_CIDR because Calico docs say that it should not be set when using kubeadm. This file sets it with v4, so it should work also with v6, but as it is not needed I did not add it. We maybe also want to create a PR to remove it from v4 section as well, but that should be in a separate PR.
There was a problem hiding this comment.
Okay, modified the calico config for dual stack as well. I am not sure how well the dual stack will work TBH, I did some fast testing but the deployment broke when setting IPv6 service CIDR in the dev env config. Control plane endpoint was v4 and it broke when trying to use v6 service CIDR at the same time. So I did not start to debug the correct variable combination for dual-stack.
There was a problem hiding this comment.
I did not add CALICO_IPV6POOL_CIDR because Calico docs say that it should not be set when using kubeadm. This file sets it with v4, so it should work also with v6, but as it is not needed I did not add it. We maybe also want to create a PR to remove it from v4 section as well, but that should be in a separate PR.
Nod, I just always added it for my IPv6/dual-stack deployments, which is indeed a bit confusing when there is no actual reason to do so such as splitting the pod CIDR into multiple IP pools with different settings. Because the Calico docs also state that there is no real need for the pool configuration variables, we should not have them unless we have a good reason to do so.
466f7ae to
e9712fc
Compare
|
/test metal3-centos-e2e-integration-test-release-1-13 metal3-dev-env-integration-test-ubuntu-main |
|
/test metal3-centos-e2e-integration-test-release-1-13 |
1 similar comment
|
/test metal3-centos-e2e-integration-test-release-1-13 |
|
/copilot-review |
|
Copilot code review has been requested by @nuhakala. Please allow a few moments for the review to be added. |
|
/copilot-review |
|
Copilot code review has been requested by @terror96. Please allow a few moments for the review to be added. |
There was a problem hiding this comment.
Pull request overview
Improves IPv6-only Metal3 deployments across networking, node initialization, Calico, and image handling.
Changes:
- Adds IPv6 firewall, service CIDR, Calico, and node-network configuration.
- Uses Crane for local registry image copies.
- Updates IPv6 documentation and removes the containerd workaround.
Reviewed changes
Copilot reviewed 15 out of 15 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
vm-setup/roles/firewall/tasks/iptables.yaml |
Adds IPv6 masquerading. |
vm-setup/roles/firewall/tasks/firewalld.yaml |
Adds libvirt-zone masquerading. |
vars.md |
Revises IPv6 setup guidance. |
tests/roles/run_tests/vars/main.yml |
Makes service CIDR configurable. |
tests/roles/run_tests/templates/main/cluster-template-workers-kubeadm-config-ubuntu.yaml |
Enables IPv6 forwarding. |
tests/roles/run_tests/templates/main/cluster-template-workers-kubeadm-config-leap.yaml |
Enables IPv6 forwarding. |
tests/roles/run_tests/templates/main/cluster-template-workers-kubeadm-config-centos.yaml |
Configures IPv6 provisioning networking. |
tests/roles/run_tests/templates/main/cluster-template-controlplane-kubeadm-config-ubuntu.yaml |
Updates IPv6 forwarding and VRRP. |
tests/roles/run_tests/templates/main/cluster-template-controlplane-kubeadm-config-leap.yaml |
Updates IPv6 networking and VRRP. |
tests/roles/run_tests/templates/main/cluster-template-controlplane-kubeadm-config-centos.yaml |
Updates IPv6 networking and VRRP. |
tests/roles/run_tests/templates/main/cluster-template-cluster.yaml |
Uses an unbracketed IPv6 API endpoint. |
tests/roles/run_tests/tasks/install_calico.yaml |
Adds IPv6 Calico configuration. |
hack/replace-containerd2.sh |
Removes the containerd replacement workaround. |
04_verify.sh |
Suppresses dynamic-function shellcheck warnings. |
02_configure_host.sh |
Copies registry images with Crane. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
e9712fc to
4766268
Compare
|
/copilot-review Also, fixing the issues I noticed that the templates don't consider dualstack case. I have not tested dualstack at all anyway, so no idea if it works otherwise. But the problem is that the templates read the IP address from the same variable in both v4 and v6 case, so configuring two addresses would require more edits. |
|
Copilot code review has been requested by @nuhakala. Please allow a few moments for the review to be added. |
- Fix image upload to local registry - Remove unneeded script hack/replace-containerd2.sh - Edit Calico configuration to work with IPv6 only - Edit templates to initialize provisioned OS correctly with IPv6 only - Add needed iptables/firewalld rules for IPv6 only - Update documentation related to IPv6 only Signed-off-by: Nuutti Hakala <nuutti.hakala@est.tech>
4766268 to
75aadb9
Compare
|
/copilot-review |
|
Copilot code review has been requested by @nuhakala. Please allow a few moments for the review to be added. |
| jump: ACCEPT | ||
| state: "{{ firewall_rule_state }}" | ||
| when: (BOOTSTRAP_CLUSTER == "kind") and (ip_stack == "v4") | ||
| when: (BOOTSTRAP_CLUSTER == "kind") and (ip_stack == "v4" or ip_stack == "v4v6") |
There was a problem hiding this comment.
dual stack is not really tested, and I don't expect it to work properly due to other problems, so I think this can be ignored
|
@Rozzii @tuminoid @elfosardo Could this get some reviews. I think these are very reasonable changes to enable IPv6 only testing. |
I have tested all three different node images (centos, ubuntu, leap) and they are working. However, I have not tested running "make" on a fresh VM, because I have limited access to IPv6 enabled VMs.