net, BGP: adjust BGP environment to use localnet NAD (#3132)

The current implementation of BGP testing is based on some knowledge
about the infrastructure: it is expected that the cluster has several
NICs, we should know the vlan of the main traffic on the cluster, as
well as for the static assignment of IP to an external router, we expect
a predefined environment variables. This PR gets rid of these
dependencies and allows us to run BGP tests without additional knowledge
about the cluster.

Main changes:
1) the usage of the NNCP and NAD localnet, the env
no longer dependent on secondary NICs.
2) dynamic receiving of IP address for an external router
from the main cluster network via DHCP.
3) a condition under which the external router and the UDN VM
are guaranteed to live on different cluster nodes, which is vital
with the current localnet configuration.

Along the way, as necessary:
1) adding `dhcp-client` to net-tools infra image
2) libs: add `namespaces` to anti-affinity and related calls
3) removing BGP sanity check as useless with the current implementation
4) de-quarantining BGP tests
5) doc fixes

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
  * Utility container now includes a DHCP client.

* **Tests**
* Re-enabled BGP connectivity tests and refactored BGP test infra to use
explicit external FRR pod state, module-scoped fixtures, and a localnet
network attachment.
* Removed legacy NIC-selection fixture and node-network-state helper
modules.

* **Behavior Changes**
* Tier2 tagging now considers BGP-marked tests when no other exclusions
apply.
  * VM anti-affinity can target specific namespaces.

* **Documentation**
* Simplified BGP test README by removing cluster requirements and env
setup.

* **Chores**
* Removed BGP environment-variable verification from network sanity
checks.

<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

Author: rnetser

conftest.py

@@ -78,7 +78,6 @@
     "node_remediation",
     "swap",
     "numa",
-    "bgp",
     "cclm",
     "mtv",
 ]

containers/utility/Dockerfile

@@ -11,6 +11,7 @@ RUN set -eux; \
     yum -y update && \
     yum install -y \
         NetworkManager \
+        dhcp-client \
         iperf3 \
         iproute \
         nftables \

libs/vm/affinity.py

@@ -15,7 +15,7 @@ def new_label(key_prefix: str) -> tuple[str, str]:
     return f"{key_prefix}-{uuid.uuid4().hex[:8]}", "true"
 
 
-def new_pod_anti_affinity(label: tuple[str, str]) -> Affinity:
+def new_pod_anti_affinity(label: tuple[str, str], namespaces: list[str] | None = None) -> Affinity:
     (key, value) = label
     return Affinity(
         podAntiAffinity=PodAntiAffinity(
@@ -25,6 +25,7 @@ def new_pod_anti_affinity(label: tuple[str, str]) -> Affinity:
                         matchExpressions=[LabelSelectorRequirement(key=key, values=[value], operator="In")]
                     ),
                     topologyKey=f"{Resource.ApiGroup.KUBERNETES_IO}/hostname",
+                    namespaces=namespaces,
                 )
             ]
         )

libs/vm/spec.py

@@ -110,6 +110,7 @@ class PodAffinityTerm:
     labelSelector: LabelSelector  # noqa: N815
     topologyKey: str  # noqa: N815
     namespaceSelector: dict[str, Any] | None = None  # noqa: N815
+    namespaces: list[str] | None = None
 
 
 @dataclass

tests/conftest.py

@@ -1014,13 +1014,6 @@ def sriov_workers(schedulable_nodes):
     yield [node for node in schedulable_nodes if node.labels.get(sriov_worker_label) == "true"]
 
 
-@pytest.fixture(scope="session")
-def vlan_base_iface(worker_node1, nodes_available_nics):
-    # Select the last NIC from the list as a way to ensure that the selected NIC
-    # is not already used (e.g. as a bond's port).
-    return nodes_available_nics[worker_node1.name][-1]
-
-
 @pytest.fixture(scope="session")
 def sriov_ifaces(sriov_nodes_states, workers_utility_pods):
     node = sriov_nodes_states[0]

tests/network/bgp/README.md

@@ -8,23 +8,3 @@ An external BGP router is required for real-world BGP connectivity, as customers
 cluster. For testing, the router is implemented as a pod running FRR within the cluster, serving the same role as an
 external router. The external BGP router (a pod in this case) must be on the same network as the cluster nodes
 to enable direct BGP sessions.
-
-### Cluster Requirements
-
-The current implementation requires:
-- Each node must have at least two NICs.
-- All secondary NICs must be connected to the same VLAN as the main IP of the `br-ex` interface.
-
-### Environment Variables
-
-Set these before running tests:
-
-- `PRIMARY_NODE_NETWORK_VLAN_TAG`: the VLAN number of the `br-ex` main IP. Setting this variable implies that a cluster
-                                   reflects the requirements mentioned above.
-- `EXTERNAL_FRR_STATIC_IPV4`: reserved IPv4 in CIDR format for the external FRR pod (e.g., 192.0.2.10/24)
-                              within the PRIMARY_NODE_NETWORK_VLAN_TAG network.
-
-```bash
-export PRIMARY_NODE_NETWORK_VLAN_TAG=<vlan_id>
-export EXTERNAL_FRR_STATIC_IPV4=<static_ip_cidr>
-```

tests/network/bgp/conftest.py

@@ -1,4 +1,3 @@
-import os
 import shlex
 from collections.abc import Generator
 from pathlib import Path
@@ -9,8 +8,8 @@
 from kubernetes.dynamic import DynamicClient
 from ocp_resources.config_map import ConfigMap
 from ocp_resources.namespace import Namespace
+from ocp_resources.network_attachment_definition import OVNOverlayNetworkAttachmentDefinition
 from ocp_resources.node import Node
-from ocp_resources.pod import Pod
 
 from libs.net import netattachdef as libnad
 from libs.net.traffic_generator import PodTcpClient as TcpClient
@@ -21,7 +20,9 @@
 from tests.network.libs import cluster_user_defined_network as libcudn
 from tests.network.libs import nodenetworkconfigurationpolicy as libnncp
 from tests.network.libs.bgp import (
+    EXTERNAL_FRR_POD_LABEL,
     POD_SECONDARY_IFACE_NAME,
+    ExternalFrrPodInfo,
     create_cudn_route_advertisements,
     create_frr_configuration,
     deploy_external_frr_pod,
@@ -31,7 +32,6 @@
 )
 from tests.network.libs.ip import random_ipv4_address
 from tests.network.libs.label_selector import LabelSelector
-from tests.network.libs.nodenetworkstate import DEFAULT_ROUTE_V4, lookup_br_ex_gateway_v4
 from tests.network.libs.vm_factory import udn_vm
 from utilities.infra import get_node_selector_dict
 
@@ -42,60 +42,44 @@
 EXTERNAL_PROVIDER_SUBNET_IPV4: Final[str] = f"{random_ipv4_address(net_seed=1, host_address=0)}/24"
 EXTERNAL_PROVIDER_IP_V4: Final[str] = f"{random_ipv4_address(net_seed=1, host_address=150)}/24"
 IPERF3_SERVER_PORT: Final[int] = 2354
+LOCALNET_NETWORK_NAME: Final[str] = "localnet-network-bgp"
 
 
-@pytest.fixture(scope="session")
-def vlan_nncp(
-    admin_client: DynamicClient,
-    vlan_base_iface: str,
-    worker_node1: Node,
+@pytest.fixture(scope="module")
+def nncp_localnet_node1(
+    admin_client: DynamicClient, worker_node1: Node
 ) -> Generator[libnncp.NodeNetworkConfigurationPolicy]:
+    desired_state = libnncp.DesiredState(
+        ovn=libnncp.OVN([
+            libnncp.BridgeMappings(
+                localnet=LOCALNET_NETWORK_NAME,
+                bridge=libnncp.DEFAULT_OVN_EXTERNAL_BRIDGE,
+                state=libnncp.BridgeMappings.State.PRESENT.value,
+            )
+        ])
+    )
     with libnncp.NodeNetworkConfigurationPolicy(
         client=admin_client,
-        name="test-vlan-nncp",
-        desired_state=libnncp.DesiredState(
-            interfaces=[
-                libnncp.Interface(
-                    name=f"{vlan_base_iface}.{os.environ['PRIMARY_NODE_NETWORK_VLAN_TAG']}",
-                    state=libnncp.NodeNetworkConfigurationPolicy.Interface.State.UP,
-                    type="vlan",
-                    vlan=libnncp.Vlan(id=int(os.environ["PRIMARY_NODE_NETWORK_VLAN_TAG"]), base_iface=vlan_base_iface),
-                )
-            ]
-        ),
+        name="localnet-nncp-bgp",
+        desired_state=desired_state,
         node_selector=get_node_selector_dict(node_selector=worker_node1.hostname),
     ) as nncp:
         nncp.wait_for_status_success()
         yield nncp
 
 
-@pytest.fixture(scope="session")
-def br_ex_gateway_v4(worker_node1: Node, admin_client: DynamicClient) -> str:
-    return lookup_br_ex_gateway_v4(node_name=worker_node1.name, client=admin_client)
-
-
-@pytest.fixture(scope="session")
-def macvlan_nad(
-    vlan_nncp: libnncp.NodeNetworkConfigurationPolicy,
-    cnv_tests_utilities_namespace: Namespace,
-    br_ex_gateway_v4: str,
+@pytest.fixture(scope="module")
+def nad_localnet(
     admin_client: DynamicClient,
-) -> Generator[libnad.NetworkAttachmentDefinition]:
-    macvlan_config = libnad.CNIPluginMacvlanConfig(
-        master=vlan_nncp.instance.spec.desiredState.interfaces[0].name,
-        ipam=libnad.IpamStatic(
-            addresses=[
-                libnad.IpamStatic.Address(address=os.environ["EXTERNAL_FRR_STATIC_IPV4"], gateway=br_ex_gateway_v4)
-            ],
-            routes=[libnad.IpamRoute(dst=DEFAULT_ROUTE_V4.dst, gw=br_ex_gateway_v4)],
-        ),
-    )
-
-    with libnad.NetworkAttachmentDefinition(
-        name="macvlan-nad-bgp",
-        namespace=cnv_tests_utilities_namespace.name,
-        config=libnad.NetConfig(name="macvlan-nad-bgp", plugins=[macvlan_config]),
+    nncp_localnet_node1: libnncp.NodeNetworkConfigurationPolicy,
+    cnv_tests_utilities_namespace: Namespace,
+):
+    with OVNOverlayNetworkAttachmentDefinition(
         client=admin_client,
+        name="localnet-nad-bgp",
+        namespace=cnv_tests_utilities_namespace.name,
+        topology="localnet",
+        network_name=LOCALNET_NETWORK_NAME,
     ) as nad:
         yield nad
 
@@ -151,8 +135,8 @@ def cudn_layer2(
                 subnets=[CUDN_SUBNET_IPV4],
             ),
         ),
-        label=APP_CUDN_LABEL,
         client=admin_client,
+        label=APP_CUDN_LABEL,
     ) as cudn:
         cudn.wait_for_status_success()
         yield cudn
@@ -171,10 +155,10 @@ def cudn_route_advertisements(
 
 
 @pytest.fixture(scope="module")
-def frr_configuration_created(admin_client: DynamicClient) -> Generator[None]:
+def frr_configuration_created(admin_client: DynamicClient, frr_external_pod: ExternalFrrPodInfo) -> Generator[None]:
     with create_frr_configuration(
         name="frr-configuration-bgp",
-        frr_pod_ipv4=os.environ["EXTERNAL_FRR_STATIC_IPV4"].split("/")[0],
+        frr_pod_ipv4=frr_external_pod.ipv4,
         external_subnet_ipv4=EXTERNAL_PROVIDER_SUBNET_IPV4,
         client=admin_client,
     ):
@@ -183,32 +167,30 @@ def frr_configuration_created(admin_client: DynamicClient) -> Generator[None]:
 
 @pytest.fixture(scope="module")
 def frr_external_pod(
-    macvlan_nad: libnad.NetworkAttachmentDefinition,
+    nad_localnet: libnad.NetworkAttachmentDefinition,
     worker_node1: Node,
     frr_configmap: ConfigMap,
     cnv_tests_utilities_namespace: Namespace,
-    br_ex_gateway_v4: str,
     admin_client: DynamicClient,
-) -> Generator[Pod]:
+) -> Generator[ExternalFrrPodInfo]:
     with deploy_external_frr_pod(
         namespace_name=cnv_tests_utilities_namespace.name,
         node_name=worker_node1.name,
-        nad_name=macvlan_nad.name,
+        nad_name=nad_localnet.name,
         frr_configmap_name=frr_configmap.name,
-        default_route=br_ex_gateway_v4,
         client=admin_client,
-    ) as pod:
+    ) as pod_info:
         # Assign a secondary IP on the secondary interface to emulate the external provider subnet
-        pod.execute(
+        pod_info.pod.execute(
             command=shlex.split(f"ip addr add {EXTERNAL_PROVIDER_IP_V4} dev {POD_SECONDARY_IFACE_NAME}"),
             container="frr",
         )
-        yield pod
+        yield pod_info
 
 
 @pytest.fixture(scope="module")
 def bgp_setup_ready(
-    frr_external_pod: Pod,
+    frr_external_pod: ExternalFrrPodInfo,
     cudn_route_advertisements: None,
     frr_configuration_created: None,
     workers: list[Node],
@@ -222,8 +204,15 @@ def vm_cudn(
     namespace_cudn: Namespace,
     cudn_layer2: libcudn.ClusterUserDefinedNetwork,
     admin_client: DynamicClient,
+    frr_external_pod: ExternalFrrPodInfo,
 ) -> Generator[BaseVirtualMachine]:
-    with udn_vm(namespace_name=namespace_cudn.name, name="vm-cudn-bgp", client=admin_client) as vm:
+    with udn_vm(
+        namespace_name=namespace_cudn.name,
+        name="vm-cudn-bgp",
+        client=admin_client,
+        template_labels=EXTERNAL_FRR_POD_LABEL,
+        anti_affinity_namespaces=[frr_external_pod.pod.namespace],
+    ) as vm:
         vm.start(wait=True)
         vm.wait_for_agent_connected()
         yield vm
@@ -239,10 +228,10 @@ def tcp_server_cudn_vm(vm_cudn: BaseVirtualMachine) -> Generator[TcpServer]:
 
 @pytest.fixture(scope="module")
 def tcp_client_external_network(
-    frr_external_pod: Pod, vm_cudn: BaseVirtualMachine, tcp_server_cudn_vm: TcpServer
+    frr_external_pod: ExternalFrrPodInfo, vm_cudn: BaseVirtualMachine, tcp_server_cudn_vm: TcpServer
 ) -> Generator[TcpClient]:
     with TcpClient(
-        pod=frr_external_pod,
+        pod=frr_external_pod.pod,
         server_ip=lookup_iface_status(vm=vm_cudn, iface_name=lookup_primary_network(vm=vm_cudn).name)[IP_ADDRESS],
         server_port=IPERF3_SERVER_PORT,
         bind_interface=EXTERNAL_PROVIDER_IP_V4.split("/")[0],

tests/network/bgp/test_bgp_connectivity.py

@@ -1,26 +1,17 @@
 import pytest
 
 from libs.net.traffic_generator import is_tcp_connection
-from utilities.constants import QUARANTINED
 from utilities.virt import migrate_vm_and_verify
 
-pytestmark = [pytest.mark.bgp, pytest.mark.usefixtures("bgp_setup_ready")]
+pytestmark = [pytest.mark.bgp, pytest.mark.ipv4, pytest.mark.usefixtures("bgp_setup_ready")]
 
 
 @pytest.mark.polarion("CNV-12276")
-@pytest.mark.xfail(
-    reason=f"{QUARANTINED}: BGP test suite infra dependencies are not met, tracked in CNV-69734",
-    run=False,
-)
 def test_connectivity_cudn_vm_and_external_network(tcp_server_cudn_vm, tcp_client_external_network):
     assert is_tcp_connection(server=tcp_server_cudn_vm, client=tcp_client_external_network)
 
 
 @pytest.mark.polarion("CNV-12281")
-@pytest.mark.xfail(
-    reason=f"{QUARANTINED}: BGP test suite infra dependencies are not met, tracked in CNV-69734",
-    run=False,
-)
 def test_connectivity_is_preserved_during_cudn_vm_migration(
     tcp_server_cudn_vm,
     tcp_client_external_network,

tests/network/conftest.py

@@ -5,7 +5,6 @@
 """
 
 import logging
-import os
 
 import pytest
 from kubernetes.dynamic import DynamicClient
@@ -340,26 +339,6 @@ def _verify_ip_family(family, is_supported_in_cluster):
             else:
                 LOGGER.info(f"Validated network lane is running against an {family} supported cluster")
 
-    def _verify_bgp_env_vars():
-        """Verify if the cluster supports running BGP tests.
-
-        Requires the following environment variables to be set:
-        PRIMARY_NODE_NETWORK_VLAN_TAG: expected VLAN number on the node br-ex interface.
-        EXTERNAL_FRR_STATIC_IPV4: reserved IP in CIDR format for the external FRR pod inside
-                                  PRIMARY_NODE_NETWORK_VLAN_TAG network.
-        """
-        if any(test.get_closest_marker("bgp") and not test.get_closest_marker("xfail") for test in collected_tests):
-            LOGGER.info("Verifying if the cluster supports running BGP tests...")
-            required_env_vars = [
-                "PRIMARY_NODE_NETWORK_VLAN_TAG",
-                "EXTERNAL_FRR_STATIC_IPV4",
-            ]
-            missing_env_vars = [var for var in required_env_vars if not os.getenv(var)]
-
-            if missing_env_vars:
-                failure_msgs.append(f"BGP tests require the following environment variables: {missing_env_vars}")
-                return
-
     def _verify_nmstate_running_pods(_admin_client, namespace):
         # TODO: Only test if nmstate is required by the test(s)
         if not namespace:
@@ -395,7 +374,6 @@ def _verify_mtv_installed():
     _verify_sriov()
     _verify_ip_family(family="ipv4", is_supported_in_cluster=ipv4_supported_cluster)
     _verify_ip_family(family="ipv6", is_supported_in_cluster=ipv6_supported_cluster)
-    _verify_bgp_env_vars()
     _verify_nmstate_running_pods(_admin_client=admin_client, namespace=nmstate_namespace)
     _verify_mtv_installed()