fix: set proxyMode to nftables on rhel10

[justinsb]

We force the use of nftables on rhel10, but kube-proxy was defaulting to iptables
and failing to start on rhel10 because of a missing kernel module (nft_ct).

The module is available in the GCE images, but not the AWS images.

Issue #17915

[justinsb]

/test ?

[justinsb]

/test pull-kops-gce-distro-rhel10

[rifelpet]

Unfortunately the presubmit jobs for different distros all use cilium including its kube-proxy replacement, so none of them run kube-proxy

https://github.com/kubernetes/test-infra/blob/feab1a5be3fa3c7ff1723ab0348fe1f00bcfea69/config/jobs/kubernetes/kops/build_jobs.py#L1423-L1426

[justinsb]

Unfortunately the presubmit jobs for different distros all use cilium including its kube-proxy replacement, so none of them run kube-proxy

I'll propose adding some more optional presubmits! (Edit: doing so in kubernetes/test-infra#36368 )

[justinsb]

/test ?

[justinsb]

/test pull-kops-aws-kindnet-rhel10arm64

[justinsb]

/test pull-kops-gce-ipalias-rhel10

Might as well

[justinsb]

OK some progress here, AWS RHEL10 passes with this PR, fails without it. One complication is that the GCE RHEL10 test passes without this PR. Trying to figure out why here: #17922 (comment)

[justinsb]

So it looks like GCE rhel10 has nft_ct loaded and AWS rhel10 does not have that module loaded: #17922 (comment)

So this PR is one way to handle it, but we could also try loading nft_ct. I feel like if we're using nftables telling kube-proxy to use nftables feels "right", but I do not have much knowledge here.

[rifelpet]

AWS is also in better shape with this PR: kubernetes/test-infra#36364

https://testgrid.k8s.io/kops-distro-rhel10#kops-grid-kindnet-rhel10arm64-k33

so i think we can be confident that --proxy-mode=nftables is the right approach here.

[justinsb]

Awesome. Do you think we should ask users to set that manually, or do you think we should set it for them if we're installing nftables?

My 2c: we should set if for them because we're managing nftables installation, and if we make the user configure something based on whether we choose to install nftables that's going to be super confusing. But very open to different thoughts here!

[justinsb]

We discussed in office hours: the plan is to force kube-proxy to nftables only on rhel10, where we are also installing the nftables packages (and not the iptables packages).

We can tweak this for other distros in future, but right now these seem to be the only distros that are failing with this error. And we want to unblock the release!

/test pull-kops-gce-ipalias-rhel10

/test pull-kops-aws-kindnet-rhel10arm64

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: hakman

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• OWNERS [hakman]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[rifelpet]

/retest