Skip to content

OSDOCS-10810: Add additional routing capabilities #89983

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 7 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions _topic_maps/_topic_map.yml
Original file line number Diff line number Diff line change
Expand Up @@ -1721,6 +1721,15 @@ Topics:
File: metallb-troubleshoot-support
- Name: Associating secondary interfaces metrics to network attachments
File: associating-secondary-interfaces-metrics-to-network-attachments
- Name: Route advertisements
Dir: route_advertisements
Topics:
- Name: About route advertisements
File: about-route-advertisements
- Name: Configuring route advertisements
File: configuring-route-advertisements
- Name: Example route advertisements setup
File: example-route-advertisement-setup
---
Name: Storage
Dir: storage
Expand Down
Binary file added images/524-openshift-bgp-ovn-k8s-no-vpn-0325.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
8 changes: 8 additions & 0 deletions modules/nw-operator-cr.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -199,6 +199,14 @@ endif::operator[]
|`object`
|Specify a configuration object for customizing network policy audit logging. If unset, the defaults audit log settings are used.

|`routeAdvertisements`
|`string`
a|Specifies whether to advertise cluster network routes. The default value is `Disabled`.
--
- `Enabled`: Apply routing configured in `RouteAdvertisements` objects.
- `Disabled`: Do not provide any additional routing capabilities.
--

|`gatewayConfig`
|`object`
|Optional: Specify a configuration object for customizing how egress traffic is sent to the node gateway.
Expand Down
66 changes: 66 additions & 0 deletions modules/nw-route-advertisements-config.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
// Module included in the following assemblies:
//
// * networking/route_advertisements/configuring-route-advertisements.adoc

:_mod-docs-content-type: PROCEDURE
[id="nw-route-advertisements-config_{context}"]
= Configuring route advertisements

As a cluster administrator, you can enable or disable additional routing support for your cluster.

[id="enabling-bgp-routing-support_{context}"]
== Enabling additional routing support

As a cluster administrator, additional routing support for your cluster.

.Prerequisites

* Install the OpenShift CLI (`oc`).
* Log in to the cluster with a user with `cluster-admin` privileges.
* Cluster is installed on bare-metal infrastructure.

.Procedure

. To enable a routing provider and additional route advertisements, enter the following command:
+
[source,terminal]
----
$ oc patch Network.operator.openshift.io cluster --type=merge \
-p='{
"spec": {
"additionalRoutingCapabilities": {
"providers": ["FRR"]
},
"defaultNetwork": {
"ovnKubernetesConfig": {
"routeAdvertisements": "Enabled"
}}}}'
----

[id="disabling-bgp-routing-support_{context}"]
== Disabling additional routing support

As a cluster administrator, you can disable additional route advertisements for your cluster.

.Prerequisites

* Install the OpenShift CLI (`oc`).
* Log in to the cluster with a user with `cluster-admin` privileges.
* Cluster is installed on bare-metal infrastructure.

.Procedure

* To disable additional routing support, enter the following command:
+
[source,terminal]
----
$ oc patch network.operator cluster -p '{
"spec": {
"defaultNetwork": {
"ovnKubernetesConfig": {
"routeAdvertisements": "Disabled"
}
}
}
}'
----
81 changes: 81 additions & 0 deletions modules/nw-routeadvertisements-about.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
// Module included in the following assemblies:
//
// * networking/route_advertisements/about-route-advertisements.adoc

:_mod-docs-content-type: CONCEPT
[id="nw-routeadvertisements-about_{context}"]
= Import and export pod network routes with Border Gateway Protocol

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this section, the title mentioned "Import", but the description only covered exporting route, need to add more description that external routes from provider network can be imported or learned into cluster


With route advertisements enabled, the OVN-Kubernetes network plugin supports directly advertising routes within the cluster and to the provider network.

For example, routes are dynamically discovered. This eliminates the need to manually configure routes on each node. Routing for cluster user-defined networks can be announced to the provider network, avoiding manual configuration steps.

[id="supported-platforms_{context}"]
== Supported platforms

Advertising routes with BGP is supported on the following infrastructure types:

- Bare-metal
- {vmw-full} on-premise

[id="prerequisites_{context}"]
== Prerequisites

To use route advertisements, you must have configured BGP for your network infrastructure. Outages or misconfigurations of your network infrastructure might cause disruptions to your cluster network.

[id="cluster-network-operator_{context}"]
== Cluster Network Operator configuration

The Cluster Network Operator (CNO) API exposes several fields to configure additional routing capabilities:

- `spec.additionalRoutingCapabilities.providers`: Specifies an additional routing provider, which is required to advertise routes. The only supported value is `FRR`. Enables deployment of the FRR-K8S daemon for the cluster. When enabled, the FRR-K8S daemon is deployed on all nodes.
- `spec.defaultNetwork.ovnKubernetesConfig.routeAdvertisements`: Enables route advertisements for the default cluster network and cluster user-defined networks. The `spec.additionalRoutingCapabilities` field must be set to `FRR` to enable this feature.

[id="compatibility-with-other-networking-features_{context}"]
== Compatibility with other networking features

Route advertisements support the following {product-title} Networking features:

Multiple external gateways::
--
When used with multiple external gateways (MEG) as implemented with the `AdminPolicyBasedExternalRoute` CR, there are several potential interactions:

- When BGP discovers routes to next hops, there can overlap with gateways configured with MEG or AdminPolicyBasedExternalRoute CRs. The behavior for this is undefined.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The phrase "When BGP discovers routes to next hops" sounds odd to me, but it might be that I'm not familiar enough with this content. Should it be "...discovers routes to the next hops"?

--

EgressIPs::
Supports the use of EgressIPs. The node where an egress IP address resides advertises the EgressIP. An egress IP address does not need to be on the same layer 2 network subnet as the egress node, because do not rely on gratuitous ARP (GARP).

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

egressIP design has changed, with latest design, egressIP address still has to be one from same subnet of egress node


Services::
Works in concert with the MetalLB Operator to advertise services to the provider network.

Egress service::
Full support.

Egress firewall::
Full support.

Egress QoS::
Full support.

Network policies::
Full support.

Direct pod ingress::
Full support for the default cluster network and user-defined networks.

[id="considerations-for-use-with-the-metallb-operator_{context}"]
== Considerations for use with the MetalLB Operator

The MetalLB Operator is installed as an add-on to the cluster. Deployment of the MetalLB Operator automatically enables FRR-K8s as an additional routing capability provider and uses the FRR-K8s daemon installed by this feature.

[id="bgp-routing-custom-resources_{context}"]
== BGP routing custom resources

The following custom resources (CRs) are used to configure route advertisements with BGP:

`RouteAdvertisements`::
This custom resource defines the advertisements for the BGP routing. From this CR the OVN-Kubernetes controller generates a `FRRConfiguration` object that configures the FRR daemon to advertise the routes. This CR is cluster scoped.

`FRRConfiguration`::
This custom resource defines the FRR configuration for BGP routing. This CR is namespaced.
Loading