ipam: Wait for ENI netlink iface before configuring ingress routes

In ENI IPAM mode the operator creates ENIs asynchronously, so at agent
startup the datapath can query netlink for the ifindex of the interface
owning a given MAC before that ENI's netlink interface has materialized.
Commit 60faea1 added waitForENI() precisely to defeat that race, but
it was only wired into the router path (reallocateRouterIPs). The ingress
path (allocateIngressIPs) calls RoutingInfo.Configure() with no such
guard, so when the ingress IP lives on a secondary ENI that has not shown
up yet, retrieveIfIndexFromMAC fails with "interface with MAC <mac> not
found". Unlike the router path, the ingress path only warns and swallows
the error, and there is no reconciler to retry it, so the missed route is
never reinstalled and the stray warning turns check-log-errors red.

Mirror the router guard into the ingress path: thread ctx into
allocateIngressIPs and, in the IPv4 ENI branch immediately before
Configure(), poll waitForENI on the ENI's PrimaryMAC. This is not
masking. If the ENI genuinely never appears, waitForENI returns after the
same bounded backoff, Configure() still fails, and the warning still
fires. Only the transient sub-8s async-creation window is absorbed, which
is exactly the window 60faea1 was written to close.

The Configure() call also runs in AlibabaCloud ENI mode, and waitForENI
just polls netlink for the MAC, so gate the wait on ENI or AlibabaCloud
rather than ENI alone, and do the same on the router path so the two stay
consistent. Raise the "interface not found" message from warn to error on
both paths, since it precedes a route configuration that is about to fail.

AIL:3
Signed-off-by: André Martins <andre@cilium.io>

Author: aanm

daemon/infraendpoints/infra_ip_allocation.go

@@ -292,12 +292,11 @@ func (r *infraIPAllocator) reallocateRouterIPs(ctx context.Context, family node.
 			return nil, fmt.Errorf("failed to create router info: %w", err)
 		}
 
-		// wait for ENI to be up and running before configuring routes and rules.
-		// This avoids spurious errors where netlink is not able to find
-		// the ifindex by its MAC because the ENI is not showing up yet.
-		if r.daemonConfig.IPAM == ipamOption.IPAMENI {
+		// Wait for the ENI to show up before configuring routes and rules, to
+		// avoid netlink failing to find the ifindex by its MAC.
+		if r.daemonConfig.IPAM == ipamOption.IPAMENI || r.daemonConfig.IPAM == ipamOption.IPAMAlibabaCloud {
 			if err := r.waitForENI(ctx, result.PrimaryMAC); err != nil {
-				r.logger.Warn("unable to find ENI netlink interface, this will likely lead to an error in configuring the router routes and rules",
+				r.logger.Error("Unable to find ENI netlink interface, this will likely lead to an error in configuring the router routes and rules",
 					logfields.MACAddr, result.PrimaryMAC,
 				)
 			}
@@ -425,7 +424,7 @@ func (r *infraIPAllocator) allocateHealthIPs(oldV4HealthIP netip.Addr, oldV6Heal
 	return nil
 }
 
-func (r *infraIPAllocator) allocateIngressIPs(oldV4IngressIP net.IP, oldV6IngressIP net.IP) error {
+func (r *infraIPAllocator) allocateIngressIPs(ctx context.Context, oldV4IngressIP net.IP, oldV6IngressIP net.IP) error {
 	if !r.daemonConfig.EnableEnvoyConfig {
 		return nil
 	}
@@ -475,6 +474,15 @@ func (r *infraIPAllocator) allocateIngressIPs(oldV4IngressIP net.IP, oldV6Ingres
 			if ingressRouting, err := r.parseRoutingInfo(result); err != nil {
 				r.logger.Warn("Unable to allocate ingress information for ENI", logfields.Error, err)
 			} else {
+				// The ingress IP may sit on a different ENI than the router IP, so
+				// wait for its ENI to show up before configuring routes and rules,
+				// to avoid netlink failing to find the ifindex by its MAC.
+				if err := r.waitForENI(ctx, result.PrimaryMAC); err != nil {
+					r.logger.Error("Unable to find ENI netlink interface, this will likely lead to an error in configuring the ingress routes and rules",
+						logfields.MACAddr, result.PrimaryMAC,
+					)
+				}
+
 				if err := ingressRouting.Configure(
 					result.IP,
 					r.mtuManager.GetDeviceMTU(),
@@ -553,7 +561,7 @@ func (r *infraIPAllocator) AllocateIPs(ctx context.Context) error {
 		return fmt.Errorf("failed to allocate service loopback IPs: %w", err)
 	}
 
-	if err := r.allocateIngressIPs(localNode.IPv4IngressIP, localNode.IPv6IngressIP); err != nil {
+	if err := r.allocateIngressIPs(ctx, localNode.IPv4IngressIP, localNode.IPv6IngressIP); err != nil {
 		return fmt.Errorf("failed to allocate ingress IPs: %w", err)
 	}