OSDOCS-14356-New: Added bond best practices info to networking docs

dfitzmau · dfitzmau · commit 759b474c8c70 · 2025-11-20T10:57:13.000Z
diff --git a/_topic_maps/_topic_map.yml b/_topic_maps/_topic_map.yml
@@ -1748,6 +1748,8 @@ Topics:
     File: verifying-connectivity-endpoint
   - Name: Changing the cluster network MTU
     File: changing-cluster-network-mtu
+  - Name: Network bonding considerations
+    File: network-bonding-considerations
   - Name: Using Stream Control Transmission Protocol
     File: using-sctp
   - Name: Associating secondary interfaces metrics to network attachments
diff --git a/installing/installing_bare_metal/ipi/ipi-install-installation-workflow.adoc b/installing/installing_bare_metal/ipi/ipi-install-installation-workflow.adoc
@@ -38,9 +38,6 @@ include::modules/creating-manifest-file-customized-br-ex-bridge.adoc[leveloffset
 // Scale each machine set to compute nodes
 include::modules/creating-scaling-machine-sets-compute-nodes-networking.adoc[leveloffset=+2]
 
-// Enabling OVS balance-slb mode for your cluster
-include::modules/enabling-OVS-balance-slb-mode.adoc[leveloffset=+1]
-
 // Establishing communication between subnets
 include::modules/ipi-install-establishing-communication-between-subnets.adoc[leveloffset=+1]
 
diff --git a/installing/installing_bare_metal/upi/installing-bare-metal-network-customizations.adoc b/installing/installing_bare_metal/upi/installing-bare-metal-network-customizations.adoc
@@ -79,9 +79,6 @@ include::modules/creating-manifest-file-customized-br-ex-bridge.adoc[leveloffset
 // Scale each machine set to compute nodes
 include::modules/creating-scaling-machine-sets-compute-nodes-networking.adoc[leveloffset=+2]
 
-// Enabling OVS balance-slb mode for your cluster
-include::modules/enabling-OVS-balance-slb-mode.adoc[leveloffset=+1]
-
 include::modules/installation-infrastructure-user-infra.adoc[leveloffset=+1]
 
 [role="_additional-resources"]
diff --git a/installing/installing_bare_metal/upi/installing-bare-metal.adoc b/installing/installing_bare_metal/upi/installing-bare-metal.adoc
@@ -100,9 +100,6 @@ include::modules/creating-manifest-file-customized-br-ex-bridge.adoc[leveloffset
 // Scale each machine set to compute nodes
 include::modules/creating-scaling-machine-sets-compute-nodes-networking.adoc[leveloffset=+2]
 
-// Enabling OVS balance-slb mode for your cluster
-include::modules/enabling-OVS-balance-slb-mode.adoc[leveloffset=+1]
-
 include::modules/installation-infrastructure-user-infra.adoc[leveloffset=+1]
 
 [role="_additional-resources"]
diff --git a/installing/installing_bare_metal/upi/installing-restricted-networks-bare-metal.adoc b/installing/installing_bare_metal/upi/installing-restricted-networks-bare-metal.adoc
@@ -94,9 +94,6 @@ include::modules/creating-manifest-file-customized-br-ex-bridge.adoc[leveloffset
 // Scale each machine set to compute nodes
 include::modules/creating-scaling-machine-sets-compute-nodes-networking.adoc[leveloffset=+2]
 
-// Enabling OVS balance-slb mode for your cluster
-include::modules/enabling-OVS-balance-slb-mode.adoc[leveloffset=+1]
-
 include::modules/installation-infrastructure-user-infra.adoc[leveloffset=+1]
 
 [role="_additional-resources"]
diff --git a/modules/configuring-localnet-switched-topology.adoc b/modules/configuring-localnet-switched-topology.adoc
@@ -23,6 +23,11 @@ When attaching a secondary network, you can either use the existing `br-ex` brid
 - If your nodes include only a single network interface, you must use the existing bridge. This network interface is owned and managed by OVN-Kubernetes and you must not remove it from the `br-ex` bridge or alter the interface configuration. If you remove or alter the network interface, your cluster network will stop working correctly.
 - If your nodes include several network interfaces, you can attach a different network interface to a new bridge, and use that for your secondary network. This approach provides for traffic isolation from your primary cluster network.
 
+[NOTE]
+====
+As a postinstallation task, you cannot make configuration changes to the `br-ex` bridge or its underlying interfaces. As a workaround, use a secondary network interface connected to your host or switch.
+====
+
 The `localnet1` network is mapped to the `br-ex` bridge in the following example:
 
 .Example mapping for sharing a bridge
diff --git a/modules/configuring-ovnk-use-second-ovs-bridge.adoc b/modules/configuring-ovnk-use-second-ovs-bridge.adoc
@@ -39,9 +39,9 @@ For more information about useful situations for the additional `br-ex1` bridge
 +
 [IMPORTANT]
 ====
-Do not use the Kubernetes NMState Operator to define or a `NodeNetworkConfigurationPolicy` (NNCP) manifest file to define the additional interface.
+Do not use the Kubernetes NMState Operator or a `NodeNetworkConfigurationPolicy` (NNCP) manifest file to define the additional interface. Ensure that the additional interface or sub-interfaces when defining a `bond` interface are not used by an existing `br-ex` OVN Kubernetes network deployment.
 
-Also ensure that the additional interface or sub-interfaces when defining a `bond` interface are not used by an existing `br-ex` OVN Kubernetes network deployment.
+As a postinstallation task, you cannot make configuration changes to the `br-ex` bridge or its underlying interfaces. As a workaround, use a secondary network interface connected to your host or switch.
 ====
 +
 .. Create the following interface definition files. These files get added to a machine configuration manifest file so that host nodes can access the definition files. 
diff --git a/modules/enabling-active-backup-mode.adoc b/modules/enabling-active-backup-mode.adoc
@@ -0,0 +1,15 @@
+// Module included in the following assemblies:
+//
+// * networking/advanced_networking/network-bonding-considerations.adoc
+
+:_mod-docs-content-type: PROCEDURE
+[id="enabling-active-backup-mode_{context}"]
+= Enabling active-backup mode for your cluster
+
+The `active-backup` mode provides fault tolerance for network connections by switching to a backup link where the primary link fails. The mode specifies the following ports for your cluster:
+
+* An active port where one physical interface sends and receives traffic at any given time.
+* A standby port where all other ports act as backup links and continously monitor their link status.
+
+During a failover process, if an active port or its link fails, the bonding logic switches all network traffic to a standby port. This standby port becomes the new active port. For failover to work, all ports in a bond must share the same Media Access Control (MAC) address. 
+
diff --git a/modules/installation-network-user-infra.adoc b/modules/installation-network-user-infra.adoc
@@ -89,17 +89,13 @@ During the initial boot, the machines require an IP address configuration that i
 
 [NOTE]
 ====
-* It is recommended to use a DHCP server for long-term management of the cluster machines. Ensure that the DHCP server is configured to provide persistent IP addresses, DNS server information, and hostnames to the cluster machines.
+* Consider using a DHCP server for long-term management of the cluster machines. Ensure that the DHCP server is configured to provide persistent IP addresses, DNS server information, and hostnames to the cluster machines.
 
 * If a DHCP service is not available for your user-provisioned infrastructure, you can instead provide the IP networking configuration and the address of the DNS server to the nodes at {op-system} install time. These can be passed as boot arguments if you are installing from an ISO image. See the _Installing {op-system} and starting the {product-title} bootstrap process_ section for more information about static IP provisioning and advanced networking options.
 ====
 endif::ibm-z[]
 
-The Kubernetes API server must be able to resolve the node names of the cluster
-machines. If the API servers and worker nodes are in different zones, you can
-configure a default DNS search zone to allow the API server to resolve the
-node names. Another supported approach is to always refer to hosts by their
-fully-qualified domain names in both the node objects and all DNS requests.
+The Kubernetes API server must be able to resolve the node names of the cluster machines. If the API servers and worker nodes are in different zones, you can configure a default DNS search zone to allow the API server to resolve the node names. Another supported approach is to always refer to hosts by their fully-qualified domain names in both the node objects and all DNS requests.
 endif::azure,gcp[]
 
 ifndef::ibm-z,azure[]
@@ -114,9 +110,7 @@ endif::ibm-z,azure[]
 [id="installation-network-connectivity-user-infra_{context}"]
 == Network connectivity requirements
 
-You must configure the network connectivity between machines to allow {product-title} cluster
-components to communicate. Each machine must be able to resolve the hostnames
-of all other machines in the cluster.
+You must configure the network connectivity between machines to allow {product-title} cluster components to communicate. Each machine must be able to resolve the hostnames of all other machines in the cluster.
 
 This section provides details about the ports that are required.
 
diff --git a/modules/installation-user-infra-machines-static-network.adoc b/modules/installation-user-infra-machines-static-network.adoc
@@ -220,7 +220,9 @@ ifndef::ibm-z-kvm[]
 
 === Bonding multiple network interfaces to a single interface
 
-Optional: You can bond multiple network interfaces to a single interface by using the `bond=` option. Refer to the following examples:
+As an optional configuration, you can bond multiple network interfaces to a single interface by using the `bond=` option. To apply this configuration to your cluster, complete the procedure steps for each node that runs on your cluster.
+
+.Procedure
 
 * The syntax for configuring a bonded interface is: `bond=<name>[:<network_interfaces>][:options]`
 +
@@ -229,26 +231,24 @@ and _options_ is a comma-separated list of bonding options. Enter `modinfo bondi
 
 * When you create a bonded interface using `bond=`, you must specify how the IP address is assigned and other
 information for the bonded interface.
-
++
 ** To configure the bonded interface to use DHCP, set the bond's IP address to `dhcp`. For example:
 +
 [source,terminal]
 ----
-bond=bond0:em1,em2:mode=active-backup
 ip=bond0:dhcp
 ----
-
++
 ** To configure the bonded interface to use a static IP address, enter the specific IP address you want and related information. For example:
 ifndef::ibm-z[]
 +
 [source,terminal]
 ----
-bond=bond0:em1,em2:mode=active-backup
 ip=10.10.10.2::10.10.10.254:255.255.255.0:core0.example.com:bond0:none
 ----
 endif::ibm-z[]
 ifdef::ibm-z[]
-
++
 [source,terminal]
 ----
 bond=bond0:em1,em2:mode=active-backup,fail_over_mac=1
@@ -287,9 +287,9 @@ ifndef::ibm-z[]
 
 === Bonding multiple SR-IOV network interfaces to a dual port NIC interface
 
-Optional: You can bond multiple SR-IOV network interfaces to a dual port NIC interface by using the `bond=` option.
+As an optional configuration, you can bond multiple SR-IOV network interfaces to a dual port NIC interface by using the `bond=` option.
 
-On each node, you must perform the following tasks:
+.Procedure
 
 ifndef::installing-ibm-power[]
 . Create the SR-IOV virtual functions (VFs) following the guidance in link:https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/9/html/configuring_and_managing_virtualization/managing-virtual-devices_configuring-and-managing-virtualization#managing-sr-iov-devices_managing-virtual-devices[Managing SR-IOV devices]. Follow the procedure in the "Attaching SR-IOV networking devices to virtual machines" section.
@@ -308,12 +308,13 @@ The following examples illustrate the syntax you must use:
 
 * When you create a bonded interface using `bond=`, you must specify how the IP address is assigned and other information for the bonded interface.
 
-** To configure the bonded interface to use DHCP, set the bond's IP address to `dhcp`. For example:
+** To configure the bonded interface to use DHCP, set the `ip` parameter to `dhcp` as demonstrated in the following example:
 +
 [source,terminal]
 ----
 bond=bond0:eno1f0,eno2f0:mode=active-backup
 ip=bond0:dhcp
+fail_over_mac=0
 ----
 
 ** To configure the bonded interface to use a static IP address, enter the specific IP address you want and related information. For example:
diff --git a/modules/nw-kernel-bonding.adoc b/modules/nw-kernel-bonding.adoc
@@ -0,0 +1,23 @@
+// Module included in the following assemblies:
+//
+// * networking/advanced_networking/network-bonding-considerations.adoc
+
+:_mod-docs-content-type: CONCEPT
+[id="nw-kernel-bonding.adoc_{context}"]
+= Kernel bonding
+
+Kernel bonding is a built-in Linux kernel function where link aggregation can exist among many Ethernet interfaces to create a single logical physical interface. Kernel bonding is the default mode if no bond interfaces depend on OVS bonds. This bonding type does not give the same level of customization as supported OVS bonding.
+
+For `kernel-bonding` mode, the bond interfaces exist outside, which means they are not in the data path, of the bridge interface. Network traffic in this mode is not sent or received on the bond interface port but instead requires additional bridging capabilities for MAC address assignment at the kernel level. 
+
+If you enabled `kernel-bonding` mode on network controller interfaces (NICs) for your nodes, you must specify a Media Access Control (MAC) address failover. This configuration prevents node communication issues with the bond interfaces, such as `eno1f0` and `eno2f0`.
+
+Red Hat supports only the following value for the `fail_over_mac` parameter:
+
+* `0`: Specifies the `none` value and this is the default value that disables MAC address failover so that all compute nodes receive the same MAC address as the bond interface. With this setting, packets might be sent to inactive nodes.
+
+Red Hat does not support the following values for the `fail_over_mac` parameter:
+
+* `1`: Specifies the `active` value and sets the MAC address of the primary bond interface to always remain the same as active nodes. If during a failover, the MAC address of a node changes, the MAC address of the bond interface changes to match the new MAC address of the node.
+* `2`: Specifies the `follow` value so that during a failover, an active node gets the MAC address of the bond interface and a formerly active node receives the MAC address of the newly active node.
+
diff --git a/modules/nw-ovs-bonding.adoc b/modules/nw-ovs-bonding.adoc
@@ -0,0 +1,20 @@
+// Module included in the following assemblies:
+//
+// * networking/advanced_networking/network-bonding-considerations.adoc
+
+:_mod-docs-content-type: CONCEPT
+[id="nw-ovs-bonding_{context}"]
+= Open vSwitch (OVS) bonding
+
+With an OVS bonding configuration, you create a single, logical interface by connecting each physical network interface controller (NIC) as a port to a specific bond. This single bond then handles all network traffic, effectively replacing the function of individual interfaces.
+
+Consider the following architectural layout for OVS bridges that interact with OVS interfaces:
+
+* A network interface uses a bridge Media Access Control (MAC) address for managing protocol-level traffic and other administrative tasks, such as IP address assignment.
+* The physical MAC addresses of physical interfaces do not handle traffic.
+* OVS handles all MAC address management at the OVS bridge level.
+
+This layout simplies bond interface management as bonds acts as data paths where centralized MAC address management happens at the OVS bridge level.
+
+For OVS bonding, you can select either `active-backup` mode or `balance-slb` mode. A bonding mode specifies the policy for how bond interfaces get used during network transmission. 
+
diff --git a/modules/virt-example-nmstate-multiple-interfaces.adoc b/modules/virt-example-nmstate-multiple-interfaces.adoc
@@ -9,6 +9,11 @@
 [role="_abstract"]
 You can create multiple interfaces in the same node network configuration policy. These interfaces can reference each other, allowing you to build and deploy a network configuration by using a single policy manifest.
 
+[IMPORTANT]
+====
+If multiple interfaces use the same default configuration, a single Network Manager connection profile activates on multiple interfaces simultaneously and this causes connections to have the same universally unique identifier (UUID). To avoid this issue, ensure that each interface has a specific configuration that is different from the default configuration.
+====
+
 The following example YAML file creates a bond that is named `bond10` across two NICs and VLAN that is named `bond10.103` that connects to the bond.
 
 [source,yaml]
diff --git a/networking/advanced_networking/network-bonding-considerations.adoc b/networking/advanced_networking/network-bonding-considerations.adoc
@@ -0,0 +1,25 @@
+:_mod-docs-content-type: ASSEMBLY
+[id="network-bonding-considerations"]
+= Network bonding considerations
+include::_attributes/common-attributes.adoc[]
+:context: network-bonding-considerations
+
+toc::[]
+
+[role="_abstract"]
+The Network bonding method, also known as _link aggregration_ or _network interface controller (NIC) teaming_, combines many network interfaces into a single, logical interface. Network bonding uses different modes to handle how network traffic distributes across bonded interfaces. Each mode provides either load balancing and fault tolerance capabilities to your network. Red Hat supports Open vSwitch (OVS) bonding and kernel bonding.
+
+// Open vSwitch (OVS) bonding
+include::modules/nw-ovs-bonding.adoc[leveloffset=+1]
+
+// Enabling active-backup mode for your cluster
+include::modules/enabling-active-backup-mode.adoc[leveloffset=+2]
+
+// Enabling OVS balance-slb mode for your cluster
+include::modules/enabling-OVS-balance-slb-mode.adoc[leveloffset=+2]
+
+// Kernel bonding
+include::modules/nw-kernel-bonding.adoc[leveloffset=+1]
+
+
+
diff --git a/networking/networking_operators/k8s-nmstate-about-the-k8s-nmstate-operator.adoc b/networking/networking_operators/k8s-nmstate-about-the-k8s-nmstate-operator.adoc
@@ -38,6 +38,11 @@ Node networking is monitored and updated by the following objects:
 `NodeNetworkConfigurationPolicy`:: Describes the requested network configuration on nodes. You update the node network configuration, including adding and removing interfaces, by applying a `NodeNetworkConfigurationPolicy` CR to the cluster.
 `NodeNetworkConfigurationEnactment`:: Reports the network policies enacted upon each node.
 
+[NOTE]
+====
+As a postinstallation task, you cannot make configuration changes to the `br-ex` bridge or its underlying interfaces. As a workaround, use a secondary network interface connected to your host or switch.
+====
+
 [id="installing-the-kubernetes-nmstate-operator-cli"]
 == Installing the Kubernetes NMState Operator
 

Original file line number	Diff line number	Diff line change
@@ -39,9 +39,9 @@ For more information about useful situations for the additional `br-ex1` bridge
`39`	`39`	`+`
`40`	`40`	`[IMPORTANT]`
`41`	`41`	`====`
`42`		-Do not use the Kubernetes NMState Operator to define or a `NodeNetworkConfigurationPolicy` (NNCP) manifest file to define the additional interface.
	`42`	+Do not use the Kubernetes NMState Operator or a `NodeNetworkConfigurationPolicy` (NNCP) manifest file to define the additional interface. Ensure that the additional interface or sub-interfaces when defining a `bond` interface are not used by an existing `br-ex` OVN Kubernetes network deployment.
`43`	`43`
`44`		-Also ensure that the additional interface or sub-interfaces when defining a `bond` interface are not used by an existing `br-ex` OVN Kubernetes network deployment.
	`44`	+As a postinstallation task, you cannot make configuration changes to the `br-ex` bridge or its underlying interfaces. As a workaround, use a secondary network interface connected to your host or switch.
`45`	`45`	`====`
`46`	`46`	`+`
`47`	`47`	`.. Create the following interface definition files. These files get added to a machine configuration manifest file so that host nodes can access the definition files.`