Skip to content

BGP evpn testing and bug fixes related to non default EVPN backbone #18358

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Jun 5, 2025
Merged
16 changes: 3 additions & 13 deletions bgpd/bgp_evpn.c
Original file line number Diff line number Diff line change
Expand Up @@ -144,11 +144,8 @@ static struct vrf_irt_node *vrf_import_rt_new(struct ecommunity_val *rt)
struct vrf_irt_node *irt;

bgp_evpn = bgp_get_evpn();
if (!bgp_evpn) {
flog_err(EC_BGP_NO_DFLT,
"vrf import rt new - evpn instance not created yet");
if (!bgp_evpn)
return NULL;
}

irt = XCALLOC(MTYPE_BGP_EVPN_VRF_IMPORT_RT,
sizeof(struct vrf_irt_node));
Expand All @@ -170,11 +167,8 @@ static void vrf_import_rt_free(struct vrf_irt_node *irt)
struct bgp *bgp_evpn = NULL;

bgp_evpn = bgp_get_evpn();
if (!bgp_evpn) {
flog_err(EC_BGP_NO_DFLT,
"vrf import rt free - evpn instance not created yet");
if (!bgp_evpn)
return;
}

hash_release(bgp_evpn->vrf_import_rt_hash, irt);
list_delete(&irt->vrfs);
Expand All @@ -197,12 +191,8 @@ static struct vrf_irt_node *lookup_vrf_import_rt(struct ecommunity_val *rt)
struct vrf_irt_node tmp;

bgp_evpn = bgp_get_evpn();
if (!bgp_evpn) {
flog_err(
EC_BGP_NO_DFLT,
"vrf import rt lookup - evpn instance not created yet");
if (!bgp_evpn)
return NULL;
}

memset(&tmp, 0, sizeof(tmp));
memcpy(&tmp.rt, rt, ECOMMUNITY_SIZE);
Expand Down
7 changes: 6 additions & 1 deletion bgpd/bgp_evpn_vty.c
Original file line number Diff line number Diff line change
Expand Up @@ -3516,7 +3516,7 @@ static void evpn_set_advertise_all_vni(struct bgp *bgp)
static void evpn_unset_advertise_all_vni(struct bgp *bgp)
{
bgp->advertise_all_vni = 0;
bgp_set_evpn(bgp_get_default());
bgp_set_evpn(NULL);
bgp_zebra_advertise_all_vni(bgp, bgp->advertise_all_vni);
bgp_evpn_cleanup_on_disable(bgp);
}
Expand Down Expand Up @@ -3739,9 +3739,14 @@ DEFUN (no_bgp_evpn_advertise_all_vni,
"Advertise All local VNIs\n")
{
struct bgp *bgp = VTY_GET_CONTEXT(bgp);
struct bgp *bgp_evpn = NULL;

if (!bgp)
return CMD_WARNING;
bgp_evpn = bgp_get_evpn();
if (!bgp_evpn || bgp_evpn != bgp)
return CMD_SUCCESS;

evpn_unset_advertise_all_vni(bgp);
return CMD_SUCCESS;
}
Expand Down
6 changes: 0 additions & 6 deletions bgpd/bgpd.c
Original file line number Diff line number Diff line change
Expand Up @@ -3515,12 +3515,6 @@ static struct bgp *bgp_create(as_t *as, const char *name,
name, bgp->as_pretty);
}

/* Default the EVPN VRF to the default one */
if (inst_type == BGP_INSTANCE_TYPE_DEFAULT && !bgp_master.bgp_evpn) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this seems like a significant change: why is it necessary? if the explicit config in some other instance works, then this is benign. and if the config change doesn't work ... then that's would be something to fix.

Copy link
Member Author

@pguibert6WIND pguibert6WIND Apr 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mjstapp , without this fix, the order of BGP instance configs matters. And this is an issue of having this lock.
About removing that, please find below the discussion I had with @Tuetuopay:

Ca va trigger pour tout le monde un souci que j'ai eu en prod et que j'ai commencé à fix.
Un RR EVPN n'est pas vtep (donc bm->bgp_evpn n'a pas de raison d'être set).
Sauf que dans le cas d'un rr, frr tente quand même d'import les routes, ne trouve pas de vrf evpn, et log en boucle

bgpd[781] [J51AF-GQ2HJ][EC 33554468] vrf import rt lookup - evpn instance not created yet

actuellement ça ne se produit que si

    1. evpn est dans une vrf non-default et
    1. frr n'a pas de default vrf de conf.

ce patch (ne pas mettre default dans bm->bgp_evpn) est à combiner avec ce fix de logging. (sur ma prod j'ai ce message des centaines de fois par seconde)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so ... that exchange isn't very helpful to me, since it's in French?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for translation, the explanation is the following one:
"
It was a RR EVPN setup made for production. Initially, there were a lot of log messages flooded.
(the ones I propose to remove). Those messages were displayed because there was no backbone defined, because 'advertise-all-vni' command was not used.

This control was a workaround to avoid having log messages.
So a default evpn backbone has been defined.

Those log message are displayed on two cases:

  • evpn is in a non default bgp instance
  • default bgp instance is not configured

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let me chime in (in English, this time).

The issue at hand is setting bm->bgp_evpn aka bgp_master.bgp_evpn blindly when creating a non-vrf BGP instance, instead of setting it only when FRR is a VTEP (i.e. advertise-all-vni enabled). It makes the whole behavior inconsistent between having a default instance and not having one.

Say, for example, we make a route-reflector out of FRR using only the non-default VRF. Such a setup does not need advertise-all-vni, and it would be detrimental as it would increase the processing done for every route passing through FRR. As a matter of fact, there are a lot of stuff that FRR tries to do when bm->bgp_evpn is not NULL, which will never succeed anyways if advertise-all-vni is not enabled.

The issue I describe in the French extract above is about log messages that Philippe also disables. The setup described above (route-reflector in a non-default vrf, with no default vrf instance) will log the bgpd[781] [J51AF-GQ2HJ][EC 33554468] vrf import rt lookup - evpn instance not created yet message for every single route received by FRR. In production, I have EVPN route-reflectors that received over 250 EVPN routes/s, which translates to this log message flooding everywhere.

The proper behavior overall is to set this field when enabling the advertise-all-vni knob, and only then. This will save work overall because my setup shows that quite a bit of the EVPN code is guarded with if (bm->bgp_evpn), and that having it set to NULL is not detrimental if the speaker is not a VTEP. As a testament to the stability of such a change: overall I have more than 50 RRs in production with this setup, and they've been rock-solid.

bgp_lock(bgp);
bm->bgp_evpn = bgp;
}

bgp_lock(bgp);

bgp->allow_martian = false;
Expand Down
42 changes: 42 additions & 0 deletions tests/topotests/bgp_evpn_rt5/ce1/frr.conf
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
interface loop141 vrf vrf-141
ip address 192.168.116.61/32
ipv6 address fd00::116/128
!
interface loop142 vrf vrf-142
ip address 192.168.126.61/32
ipv6 address fd00::126/128
!
interface eth-r2.141 vrf vrf-141
ip address 192.168.106.61/24
!
interface eth-r2.142 vrf vrf-142
ip address 192.168.105.61/24
!
router bgp 65001 vrf vrf-142
bgp router-id 192.168.105.61
bgp log-neighbor-changes
no bgp ebgp-requires-policy
neighbor 192.168.105.41 remote-as 65000
neighbor 192.168.105.41 capability extended-nexthop
address-family ipv4 unicast
network 192.168.126.61/32
exit-address-family
address-family ipv6 unicast
neighbor 192.168.105.41 activate
network fd00::126/128
exit-address-family
!
router bgp 65001 vrf vrf-141
bgp router-id 192.168.106.61
bgp log-neighbor-changes
no bgp ebgp-requires-policy
neighbor 192.168.106.41 remote-as 65000
neighbor 192.168.106.41 capability extended-nexthop
address-family ipv4 unicast
network 192.168.116.61/32
exit-address-family
address-family ipv6 unicast
neighbor 192.168.106.41 activate
network fd00::116/128
exit-address-family
!
Original file line number Diff line number Diff line change
@@ -0,0 +1,127 @@
{
"65000:2":{
"rd":"65000:2",
"[5]:[0]:[32]:[192.168.116.61]":{
"prefix":"[5]:[0]:[32]:[192.168.116.61]",
"prefixLen":352,
"paths":[
{
"valid":true,
"bestpath":true,
"selectionReason":"First path received",
"pathFrom":"internal",
"routeType":5,
"ethTag":0,
"ipLen":32,
"ip":"192.168.116.61",
"metric":0,
"locPrf":100,
"weight":0,
"peerId":"192.168.1.101",
"path":"65001",
"origin":"IGP",
"nexthops":[
{
"ip":"192.168.2.2",
"hostname":"rr",
"afi":"ipv4",
"used":true
}
]
}
]
},
"[5]:[0]:[32]:[192.168.126.61]":{
"prefix":"[5]:[0]:[32]:[192.168.126.61]",
"prefixLen":352,
"paths":[
{
"valid":true,
"bestpath":true,
"selectionReason":"First path received",
"pathFrom":"internal",
"routeType":5,
"ethTag":0,
"ipLen":32,
"ip":"192.168.126.61",
"metric":0,
"locPrf":100,
"weight":0,
"peerId":"192.168.1.101",
"path":"65001",
"origin":"IGP",
"nexthops":[
{
"ip":"192.168.2.2",
"hostname":"rr",
"afi":"ipv4",
"used":true
}
]
}
]
},
"[5]:[0]:[128]:[fd00::116]":{
"prefix":"[5]:[0]:[128]:[fd00::116]",
"prefixLen":352,
"paths":[
{
"valid":true,
"bestpath":true,
"selectionReason":"First path received",
"pathFrom":"internal",
"routeType":5,
"ethTag":0,
"ipLen":128,
"ip":"fd00::116",
"metric":0,
"locPrf":100,
"weight":0,
"peerId":"192.168.1.101",
"path":"65001",
"origin":"IGP",
"nexthops":[
{
"ip":"192.168.2.2",
"hostname":"rr",
"afi":"ipv4",
"used":true
}
]
}
]
},
"[5]:[0]:[128]:[fd00::126]":{
"prefix":"[5]:[0]:[128]:[fd00::126]",
"prefixLen":352,
"paths":[
{
"valid":true,
"bestpath":true,
"selectionReason":"First path received",
"pathFrom":"internal",
"routeType":5,
"ethTag":0,
"ipLen":128,
"ip":"fd00::126",
"metric":0,
"locPrf":100,
"weight":0,
"peerId":"192.168.1.101",
"path":"65001",
"origin":"IGP",
"nexthops":[
{
"ip":"192.168.2.2",
"hostname":"rr",
"afi":"ipv4",
"used":true
}
]
}
]
}
},
"numPrefix":14,
"totalPrefix":14
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
{
"65000:2":{
"rd":"65000:2",
"[5]:[0]:[32]:[192.168.116.61]":{
"prefix":"[5]:[0]:[32]:[192.168.116.61]",
"prefixLen":352,
"paths":[
{
"valid":true,
"bestpath":true,
"selectionReason":"First path received",
"pathFrom":"internal",
"routeType":5,
"ethTag":0,
"ipLen":32,
"ip":"192.168.116.61",
"metric":0,
"locPrf":100,
"weight":0,
"peerId":"192.168.1.101",
"path":"65001",
"origin":"IGP",
"nexthops":[
{
"ip":"192.168.2.2",
"hostname":"rr",
"afi":"ipv4",
"used":true
}
]
}
]
},
"[5]:[0]:[128]:[fd00::126]":{
"prefix":"[5]:[0]:[128]:[fd00::126]",
"prefixLen":352,
"paths":[
{
"valid":true,
"bestpath":true,
"selectionReason":"First path received",
"pathFrom":"internal",
"routeType":5,
"ethTag":0,
"ipLen":128,
"ip":"fd00::126",
"metric":0,
"locPrf":100,
"weight":0,
"peerId":"192.168.1.101",
"path":"65001",
"origin":"IGP",
"nexthops":[
{
"ip":"192.168.2.2",
"hostname":"rr",
"afi":"ipv4",
"used":true
}
]
}
]
}
},
"numPrefix":10,
"totalPrefix":10
}
33 changes: 33 additions & 0 deletions tests/topotests/bgp_evpn_rt5/r2/frr.conf
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,35 @@ interface eth-rr
!
ip route 0.0.0.0/0 192.168.2.101
!
interface eth-ce1.141 vrf vrf-141
ip address 192.168.106.41/24
!
interface eth-ce1.142 vrf vrf-142
ip address 192.168.105.41/24
!
router bgp 65000 vrf vrf-142
bgp router-id 192.168.105.41
bgp log-neighbor-changes
no bgp ebgp-requires-policy
neighbor 192.168.105.61 remote-as 65001
neighbor 192.168.105.61 shutdown
neighbor 192.168.105.61 capability extended-nexthop
address-family ipv6 unicast
neighbor 192.168.105.61 activate
exit-address-family
!
router bgp 65000 vrf vrf-141
bgp router-id 192.168.106.41
bgp log-neighbor-changes
!
no bgp ebgp-requires-policy
neighbor 192.168.106.61 remote-as 65001
neighbor 192.168.106.61 shutdown
neighbor 192.168.106.61 capability extended-nexthop
address-family ipv6 unicast
neighbor 192.168.106.61 activate
exit-address-family
!
router bgp 65000
bgp router-id 192.168.0.2
bgp log-neighbor-changes
Expand All @@ -45,10 +74,14 @@ router bgp 65000 vrf vrf-101
bgp log-neighbor-changes
no bgp network import-check
address-family ipv4 unicast
import vrf vrf-141
import vrf vrf-142
network 10.0.101.2/32
network 10.0.101.12/32
exit-address-family
address-family ipv6 unicast
import vrf vrf-141
import vrf vrf-142
network fd01::2/128
network fd01::12/128
exit-address-family
Expand Down
Loading
Loading