Skip to content

Commit 4025e1a

Browse files
cumulusAniacumulusAnia
committed
Merge branch '514-features' into stage
2 parents d127ae0 + 9710fc6 commit 4025e1a

File tree

3 files changed

+190
-6
lines changed

3 files changed

+190
-6
lines changed

‎content/cumulus-linux-514/Layer-1-and-Switch-Ports/Quality-of-Service/RDMA-over-Converged-Ethernet-RoCE.md‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -62,6 +62,17 @@ cumulus@switch:~$ nv set qos roce mode lossy
6262
cumulus@switch:~$ nv config apply
6363
```
6464

65+
## MRC QoS Profile
66+
67+
The MRC QoS profile `lossy-multi-tc` enables packet trimming:
68+
69+
```
70+
cumulus@switch:~$ nv set qos roce mode lossy-multi-tc
71+
cumulus@switch:~$ nv config apply
72+
```
73+
74+
For more information about MRC, refer to {{<link url="Quality-of-Service/#mrc-packet-trimming" text="MRC Packet Trimming">}}.
75+
6576
## Single Shared Buffer Pool
6677

6778
By default, Cumulus Linux separates lossy and lossless traffic into different dedicated buffer pools on both ingress and egress. You can configure the switch to combine lossy and lossless traffic on the same ingress buffer pool for better load absorption of lossless traffic.

‎content/cumulus-linux-514/Layer-1-and-Switch-Ports/Quality-of-Service/_index.md‎

Lines changed: 174 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -943,6 +943,180 @@ cos_egr_queue.cos_7.uc = 7
943943
{{< /tab >}}
944944
{{< /tabs >}}
945945

946+
## MRC
947+
948+
<span class="a-tooltip">[MRC](## "Multipath Reliable Connection")</span> is an improvement over RoCEv2 to enhance performance in lossy environments and extend the RC transport for scalability and performance for AI and <span class="a-tooltip">[ML](## "machine learning")</span> applications over lossy networks. Some of these enhancements include allowing packets to be transmitted over multiple logical paths in the network and rapid detection and retransmission of delayed, unacknowledged, and trimmed packets.
949+
950+
MRC on Cumulus Linux supports:
951+
- SRv6 uSID support with uN (END) Endpoint behavior through static configuration
952+
- Packet trimming
953+
- Asymmetric packet trimming
954+
955+
### Configure MRC with Default Settings
956+
957+
To configure MRC to use the default settings for packet trimming and SRv6:
958+
- Set the MRC QoS profile.
959+
- Enable SRv6.
960+
- Configure the SRv6 endpoint by setting a locator and a uSID associated with the locator.
961+
962+
```
963+
cumulus@switch:~$ nv set router segment-routing srv6 state enabled
964+
cumulus@switch:~$ nv set qos roce mode lossy-multi-tc
965+
cumulus@switch:~$ nv set router segment-routing srv6 locator LOC2 block-length 32
966+
cumulus@switch:~$ nv set router segment-routing srv6 locator LOC2 func-length 0
967+
cumulus@switch:~$ nv set router segment-routing srv6 locator LOC2 node-length 16
968+
cumulus@switch:~$ nv set router segment-routing srv6 locator LOC2 prefix fcbb:bbbb:2::/48
969+
cumulus@switch:~$ nv set router segment-routing srv6 static-sid fcbb:bbbb:2::/48 behavior uN
970+
cumulus@switch:~$ nv set router segment-routing srv6 static-sid fcbb:bbbb:2::/48 locator-name LOC2
971+
cumulus@switch:~$ nv config apply
972+
```
973+
974+
{{%notice note%}}
975+
Cumulus Linux only supports an SRv6 block and node length of F3216 and a behavior of `uN` (END).
976+
{{%/notice%}}
977+
978+
To show the default QoS `lossy-multi-tc` profile settings, run the `nv show qos roce` command:
979+
980+
```
981+
cumulus@switch:~$ nv show qos roce
982+
operational applied
983+
------------------ -------------- --------------
984+
enable on on
985+
mode lossy-multi-tc lossy-multi-tc
986+
pfc
987+
pfc-priority -
988+
congestion-control
989+
congestion-mode ECN
990+
enabled-tc 1,2,3
991+
min-threshold 159.18 KB
992+
max-threshold 237.30 KB
993+
probability 100
994+
trust
995+
trust-mode pcp,dscp
996+
997+
RoCE PCP/DSCP->SP mapping configurations
998+
===========================================
999+
pcp dscp
1000+
switch-prio
1001+
- --- -------------------------------------------------------------------
1002+
------------ -----------
1003+
0 0
1004+
0,7,8,9,10,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63
1005+
0
1006+
1 1 1,2
1007+
1
1008+
2 2 3,4
1009+
2
1010+
3 3 5,6
1011+
3
1012+
4 4 11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30
1013+
4
1014+
5 5 31,32,33,34,35,36,37,38,39,40
1015+
5
1016+
6 6 -
1017+
6
1018+
7 7 -
1019+
7
1020+
1021+
RoCE SP->TC mapping and ETS configurations
1022+
=============================================
1023+
1024+
switch-prio traffic-class scheduler-weight
1025+
- ----------- ------------- ----------------
1026+
0 0 0 DWRR-4%
1027+
1 1 1 DWRR-8%
1028+
2 2 2 DWRR-18%
1029+
3 3 3 DWRR-22%
1030+
4 4 4 DWRR-22%
1031+
5 5 5 DWRR-22%
1032+
6 6 6 DWRR-4%
1033+
7 7 7 DWRR-0%
1034+
1035+
RoCE pool config
1036+
===================
1037+
name mode size switch-priorities traffic-class
1038+
- --------------------- ------- ---- ----------------- ---------------
1039+
0 lossy-default-ingress Dynamic 100% 0,1,2,3,4,5,6,7 -
1040+
2 lossy-default-egress Dynamic 100% - Exception List
1041+
================
1042+
```
1043+
1044+
SRv6 endpoints are installed as IPv6 routes into the RIB and FIB. To show SRv6 endpoints, view the
1045+
IPv6 RIB with the `nv show vrf <vrf> router rib ipv6 route` command.
1046+
1047+
```
1048+
cumulus@switch:~$ nv show vrf default router rib ipv6 route
1049+
routeFlags - * - selected, q - queued,
1050+
o - offloaded, i - installed, S - fib-selected, x – failed
1051+
Route Protocol Distance Uptime NHGId Metric Flags-
1052+
--------------- --------- -------- -------------------- ----- ------ -----
1053+
3ffe::1:0/112 connected 0 2025-04-30T20:36:32Z 315 0
1054+
*Sio3ffe::1:1/128 local 0 2025-04-30T20:36:32Z 315 0
1055+
*Sio3ffe::11:0/112 connected 0 2025-04-30T20:36:25Z 234 0
1056+
*Sio3ffe::11:1/128 local 0 2025-04-30T20:36:25Z 234 0
1057+
*Sio3ffe::22:0/112 connected 0 2025-04-30T20:36:27Z 311 0
1058+
*Sio3ffe::22:1/128 local 0 2025-04-30T20:36:27Z 311 0
1059+
*Sio::/0 static 1 2025-04-30T20:36:25Z 237 0
1060+
*Sifcbb:bbbb:2::/48 static 1 2025-04-30T21:56:00Z 319 0 *Si
1061+
```
1062+
1063+
You can view a specific route with the `nv show vrf <vrf> router rib ipv6 route <route-id>` command.
1064+
1065+
### Packet Trimming
1066+
1067+
If you do not want to use the MRC QoS profile `lossy-multi-tc` to enable packet trimming with the recommended QoS settings as shown above, you can configure the packet trimming settings you want to use.
1068+
1069+
{{%notice note%}}
1070+
If you set `qos roce mode lossy-multi-tc`, you do not need to configure the packet trimming settings.
1071+
{{%/notice%}}
1072+
1073+
To configure packet trimming:
1074+
- Set the packet trimming profile to `packet-trim-default`.
1075+
- Set the forwarding port used for recirculating the trimmed packets to egress the interface (NVIDIA Spectrum-4 switch only). If you do not configure a service port, Cumulus Linux uses the last service port in on the switch.
1076+
- Set the maximum size of the trimmed packet.
1077+
- Set the DSCP value to be marked on the trimmed packets.
1078+
- Egress port and traffic-class from which dropped traffic is trimmed.
1079+
- Set the egress traffic class on which to send the trimmed packet.
1080+
- Enable packet trimming.
1081+
1082+
```
1083+
cumulus@switch:~$ nv set system forwarding packet-trim profile packet-trim-default
1084+
cumulus@switch:~$ nv set system forwarding packet-trim service-port swp65
1085+
cumulus@switch:~$ nv set system forwarding packet-trim remark dscp 10
1086+
cumulus@switch:~$ nv set system forwarding packet-trim size 128
1087+
cumulus@switch:~$ nv set system forwarding packet-trim traffic-class 4
1088+
cumulus@switch:~$ nv set interface swp1-3 packet-trim egress-eligibility traffic-class 1
1089+
cumulus@switch:~$ nv set system forwarding packet-trim state enabled
1090+
cumulus@switch:~$ nv config apply
1091+
```
1092+
1093+
To show packet trimming configuration, run the `nv show system forwarding packet-trim` command:
1094+
1095+
```
1096+
cumulus@switch:~$ nv show system forwarding packet-trim
1097+
applied
1098+
------------- -----------
1099+
remark
1100+
dscp 10
1101+
state enabled
1102+
size 128
1103+
profile adaptive-rc
1104+
traffic-class 4
1105+
service-port swp65
1106+
1107+
eligiblity-egress-interface-tc
1108+
=================================
1109+
Interface TC
1110+
--------- --
1111+
swp1 1
1112+
2
1113+
swp2 1
1114+
2
1115+
```
1116+
1117+
### Asymmetric Packet Trimming
1118+
1119+
9461120
## Egress Scheduler
9471121

9481122
Cumulus Linux supports 802.1Qaz, Enhanced Transmission Selection, which allows the switch to assign bandwidth to egress queues and then schedule the transmission of traffic from each queue. 802.1Qaz supports Priority Queuing.

‎content/cumulus-linux-514/Whats-New/_index.md‎

Lines changed: 5 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -30,12 +30,11 @@ Cumulus Linux 5.14.0 supports new platforms, provides bug fixes, and contains se
3030
- gNMI requirements phase 4
3131
- FIPS 140-2 and 3 approved crypto modules
3232
- Crypto mechanisms to prevent unauthorized modification of all information
33-
- Strong authenticators
34-
- MRC packet trimming
35-
- MRC SRv6 subset to support OAI
36-
- New QoS profile to support MRC
37-
- Packet trimming with asymmetric DSCP
38-
- SRv6 counters
33+
- Strong authenticators
34+
- MRC:
35+
- {{<link url="Quality-of-Service/#configure-mrc-with-default-settings" text="Packet trimming and SRv6">}}
36+
- {{<link url="RDMA-over-Converged-Ethernet-RoCE/#mrc-qos-profile" text="New QoS profile for MRC">}}
37+
- {{<link url="Quality-of-Service/#asymmetric-packet-trimming" text="Packet trimming with asymmetric DSCP">}}
3938
- NVUE
4039
- {{< expand "Changed NVUE Commands" >}}
4140
| Cumulus Linux 5.14 | Cumulus Linux 13 and Earlier |

0 commit comments

Comments
 (0)