Skip to content

OSDOCS-10810: Add BGP routing #88826

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 12 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 10 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions _topic_maps/_topic_map.yml
Original file line number Diff line number Diff line change
Expand Up @@ -1721,6 +1721,15 @@ Topics:
File: metallb-troubleshoot-support
- Name: Associating secondary interfaces metrics to network attachments
File: associating-secondary-interfaces-metrics-to-network-attachments
- Name: BGP routing
Dir: bgp_routing
Topics:
- Name: About BGP routing
File: about-bgp-routing
- Name: Configuring BGP routing
File: configuring-bgp-routing
- Name: Migrating FRR-K8s custom resources
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe the project name has a lowercase "k": FRR-k8s

We should refer to it consistently and accurately wherever it appears. 🙂

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kowen-rh, wasn't really familiar with FRR-K8s, but the original documentation work in this area is: modules/nw-metallb-frr-k8s-merge-multiple-configurations.adoc and that does capitalize K, so I've been maintaining consistency with this original work.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That makes sense to me as long as it's consistent. 🙂

File: migrating-frr-k8s-resources
---
Name: Storage
Dir: storage
Expand Down
49 changes: 49 additions & 0 deletions modules/nw-bgp-about.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
// Module included in the following assemblies:
//
// * networking/bgp_routing/about-bgp-routing.adoc

:_mod-docs-content-type: CONCEPT
[id="nw-bgp-about_{context}"]
= About Border Gateway Protocol (BGP) routing

{product-title} supports BGP routing through FRRouting (FRR), a free, open source internet routing protocol suite for Linux and UNIX platforms. FRR-K8s is a Kubernetes based daemon set that exposes a subset of the FRR API in a Kubernetes-compliant manner. As a cluster administrator, you can use the FRRConfiguration custom resource (CR) to access FRR services.

[id="supported-platforms_{context}"]
== Supported platforms

BGP routing is supported on the following infrastructure types:

- Bare metal
- {vmw-full} on-premise

[id="prerequisites_{context}"]
== Prerequisites
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This isn't a procedure, so we don't need a section for prerequisites. I'd remove these two lines so that this is at the end of the "Supported platforms" section.


Use of BGP routing requires that you have properly configured BGP for your network provider. Outages or misconfigurations of your network provider might cause disruptions to your cluster network.

[id="considerations-for-use-with-the-metallb-operator_{context}"]
== Considerations for use with the MetalLB Operator

The MetalLB Operator is installed as an add-on to the cluster. Deployment of the MetalLB Operator automatically enables FRR-K8s as an additional routing capability provider and uses the FRR-K8s daemon installed by this feature.

Before upgrading to 4.18, any existing FRRConfiguration in the `metallb-system` namespace not managed by the MetalLB operator (added by a cluster administrator or any other component) needs to be copied to the `openshift-frr-k8s` namespace manually, creating the namespace if necessary.

[IMPORTANT]
====
If you are using the MetalLB Operator and there are existing `FRRConfiguration` CRs in the `metallb-system` namespace created by cluster administrators or third-party cluster components other than MetalLB Operator, you must ensure that they are copied to the `openshift-frr-k8s` namespace or that those third-party cluster components use the new namespace.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This sentence is long and complicated to read. Can we rephrase this in a simpler way or split it into multiple sentences?

====

[id="cluster-network-operator_{context}"]
== Cluster Network Operator configuration

The Cluster Network Operator API exposes the following API field to configure BGP routing:

- `spec.additionalRoutingCapabilities`: Enables deployment of the FRR-K8S daemon for the cluster, which can be used independently of route advertisements. When enabled, the FRR-K8S daemon is deployed on all nodes.

[id="bgp-routing-custom-resources_{context}"]
== BGP routing custom resources

The following custom resources are used to configure BGP routing:

`FRRConfiguration`::
This custom resource defines the FRR configuration for the BGP routing. This CR is namespaced.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a comment, but it seems like this module serves as a "miscellaneous information" page. Be careful with those, as they can become unwieldy as time progresses and more is added. It might be worth rethinking how this information is organized so that related concepts are grouped together.

55 changes: 55 additions & 0 deletions modules/nw-bgp-frr-k8s-migration.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
// Module included in the following assemblies:
//
// * networking/bgp_routing/migrating-frr-k8s-resources.adoc

:_mod-docs-content-type: CONCEPT
[id="nw-bgp-frr-k8s-migration_{context}"]
= Migrating FRR-K8s resources

You can migrate the FRR-K8s custom resources from the `metallb-system` namespace to the `openshift-frr-k8s` namespace.

.Prerequisites

* Install the OpenShift CLI (`oc`).
* Log in to the cluster with a user with `cluster-admin` privileges.

.Procedure

When upgrading from an earlier version of {product-title} with the Metal LB Operator deployed, you must manually migrate your custom frr-k8s configurations from the `metallb-system` namespace to the `openshift-frr-k8s` namespace. To move these CRs, enter the following commands:

. To create the `openshift-frr-k8s` namespace, enter the following command:
+
[source,terminal]
----
$ oc create namespace openshift-frr-k8s

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we need to create the namespace here? I thought the namespace is created automatically when metalLB operator is upgraded to 4.19, @asood-rh correct me if it's not

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@zhaozhanqi We are copying over custom FRR configurations before upgrading that is why we need to manually create the namespace. If there were no custom FRR configuration just plain upgrade of metallb operator, the namespace will be created by metallb install.
Sorry I missed your question.

----

. To automate the migration, create a `migrate.sh` with the following content:
+
[source,bash]
----
#!/bin/bash
OLD_NAMESPACE="metallb-system"
NEW_NAMESPACE="openshift-frr-k8s"
FILTER_OUT="metallb-"
oc get frrconfigurations.frrk8s.metallb.io -n "${OLD_NAMESPACE}" -o json |\
jq -r '.items[] | select(.metadata.name | test("'"${FILTER_OUT}"'") | not)' |\
jq -r '.metadata.namespace = "'"${NEW_NAMESPACE}"'"' |\
oc create -f -
----

. To execute the migration, run the following command:
+
[source,terminal]
----
$ bash migrate.sh
----

. To confirm that the migration succeeded, run the following command:
+
[source,terminal]
----
$ oc get frrconfigurations.frrk8s.metallb.io -n openshift-frr-k8s
----

After the migration is complete, you can remove the FRR-K8s custom resources from the `metallb-system` namespace.
59 changes: 59 additions & 0 deletions modules/nw-bgp-routing-config.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
// Module included in the following assemblies:
//
// * networking/bgp_routing/configuring-bgp-routing.adoc

:_mod-docs-content-type: PROCEDURE
[id="nw-bgp-routing-config_{context}"]
= Configuring Border Gateway Protocol (BGP) routing

As a cluster administrator, you can enable or disable Border Gateway Protocol (BGP) routing support for your cluster on bare-metal infrastructure.

If you are using BGP routing in conjunction with the MetalLB Operator, the necessary BGP routing support is enabled automatically. You do not need to manually enable BGP routing support.

[id="enabling-bgp-routing-support_{context}"]
== Enabling BGP routing support

As a cluster administrator, you can enable BGP routing support for your cluster.

.Prerequisites

* Install the OpenShift CLI (`oc`).
* Log in to the cluster with a user with `cluster-admin` privileges.
* Cluster is installed on compatible infrastructure.

.Procedure

* To enable a dynamic routing provider, enter the following command:
+
[source,terminal]
----
$ oc patch network.operator cluster -p '{
"spec": {
"additionalRoutingCapabilities": ["FRR"]
}
}'
----

[id="disabling-bgp-routing-support_{context}"]
== Disabling BGP routing support

As a cluster administrator, you can disable BGP routing support for your cluster.

.Prerequisites

* Install the OpenShift CLI (`oc`).
* Log in to the cluster with a user with `cluster-admin` privileges.
* Cluster is installed on compatible infrastructure.

.Procedure

* To disable dynamic routing, enter the following command:
+
[source,terminal]
----
$ oc patch network.operator cluster -p '{
"spec": {
"additionalRoutingCapabilities": null
}
}'
----
16 changes: 16 additions & 0 deletions modules/nw-operator-cr.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -110,6 +110,22 @@ endif::operator[]
|`object`
|Configures the network plugin for the cluster network.

|`spec.additionalRoutingCapabilities.providers`
|`array`
|This setting enables a dynamic routing provider. The FRR routing capability provider is required for the route advertisement feature. The only supported value is `FRR`.

--
- `FRR`: The FRR routing provider
--

[source,yaml]
----
spec:
additionalRoutingCapabilities:
providers:
- FRR
----

|`spec.kubeProxyConfig`
|`object`
|
Expand Down
1 change: 1 addition & 0 deletions networking/bgp_routing/_attributes
20 changes: 20 additions & 0 deletions networking/bgp_routing/about-bgp-routing.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
:_mod-docs-content-type: ASSEMBLY
[id="about-bgp-routing"]
= About BGP routing
include::_attributes/common-attributes.adoc[]
:context: about-bgp-routing

toc::[]

This feature provides native Border Gateway Protocol (BGP) routing capabilities for the cluster.

If you are using the MetalLB Operator and there are existing `FRRConfiguration` CRs in the `metallb-system` namespace created by cluster administrators or third-party cluster components other than the MetalLB Operator, you must ensure that they are copied to the `openshift-frr-k8s` namespace or that those third-party cluster components use the new namespace. For more information, see xref:../../networking/bgp_routing/migrating-frr-k8s-resources.adoc#migrating-frr-k8s-resources[Migrating FRR-K8s resources].

include::modules/nw-bgp-about.adoc[leveloffset=+1]
include::modules/nw-metallb-frr-k8s-configuration-crd.adoc[leveloffset=+1]

[role="_additional-resources"]
[id="additional-resources_about-bgp-routing"]
== Additional resources

- link:https://docs.frrouting.org/en/latest/bgp.html[FRRouting User Guide: BGP]
11 changes: 11 additions & 0 deletions networking/bgp_routing/configuring-bgp-routing.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
:_mod-docs-content-type: ASSEMBLY
[id="configuring-bgp-routing"]
= Configuring BGP routing
include::_attributes/common-attributes.adoc[]
:context: configuring-bgp-routing

toc::[]

As a cluster administrator, you can configure OVN-Kubernetes Border Gateway Protocol (BGP) routing support for your cluster.

include::modules/nw-bgp-routing-config.adoc[leveloffset=+1]
1 change: 1 addition & 0 deletions networking/bgp_routing/images
11 changes: 11 additions & 0 deletions networking/bgp_routing/migrating-frr-k8s-resources.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
:_mod-docs-content-type: ASSEMBLY
[id="migrating-frr-k8s-resources"]
= Migrating FRR-K8s resources
include::_attributes/common-attributes.adoc[]
:context: migrating-frr-k8s-resources

toc::[]

All user-created FRR-K8s custom resources (CRs) in the `metallb-system` namespace under {product-title} 4.17 and earlier releases must be migrated to the `openshift-frr-k8s` namespace. As a cluster administrator, complete the steps in this procedure to migrate your FRR-K8s custom resources.

include::modules/nw-bgp-frr-k8s-migration.adoc[leveloffset=+1]
1 change: 1 addition & 0 deletions networking/bgp_routing/modules
1 change: 1 addition & 0 deletions networking/bgp_routing/snippets