Skip to content

Commit f6e2b2a

Browse files
sraviteja-makerSimone Ferlin-Reiteramorenoz
authored
[PERFSCALE-4463] Add SRIOV support for node-density-cni workloads to capture SRIOV RTNL Issues (#412)
* [PERFSCALE-4463] Add NUM_SRIOVS,SRIOV_NET_DEVICEPOOL,SRIOV_NETWORK_NAME vars in node-density-cni to support SRIOV workloads Signed-off-by: Raviteja Sahukari <rsahukar@redhat.com> Co-authored-by: Simone Ferlin-Reiter <sferlin@redhat.com> Co-authored-by: Adrian Moreno Zapata <amorenoz@redhat.com> * [PERFSCALE-4463] Add SRIOV Network manifest for IPAM Configuration Signed-off-by: Raviteja Sahukari <rsahukar@redhat.com> Co-authored-by: Simone Ferlin-Reiter <sferlin@redhat.com> Co-authored-by: Adrian Moreno Zapata <amorenoz@redhat.com> * [PERFSCALE-4463] Add annotations to workload deployment manifests to support SRIOV and Performance profile Signed-off-by: Raviteja Sahukari <rsahukar@redhat.com> Co-authored-by: Simone Ferlin-Reiter <sferlin@redhat.com> Co-authored-by: Adrian Moreno Zapata <amorenoz@redhat.com> * [PERFSCALE-4463] Update node-density-cni config file to support SRIOV workloads Signed-off-by: Raviteja Sahukari <rsahukar@redhat.com> Co-authored-by: Simone Ferlin-Reiter <sferlin@redhat.com> Co-authored-by: Adrian Moreno Zapata <amorenoz@redhat.com> * [PERFSCALE-4463] Resolved undefined variables for node-density-cni SRIOV workloads Signed-off-by: Raviteja Sahukari <rsahukar@redhat.com> * [PERFSCALE-4463] Update the reference variables to resolve rendering errors Signed-off-by: Raviteja Sahukari <rsahukar@redhat.com> * [PERFSCALE-4463] Update README.md how to use node-density-cni with SRIOV VFs workload Signed-off-by: Raviteja Sahukari <rsahukar@redhat.com> * [PERFSCALE-4463] Resolve the object verfication error by creating a pre-manual execution step for SRIOV network creation Signed-off-by: Raviteja Sahukari <rsahukar@redhat.com> --------- Signed-off-by: Raviteja Sahukari <rsahukar@redhat.com> Co-authored-by: Simone Ferlin-Reiter <sferlin@redhat.com> Co-authored-by: Adrian Moreno Zapata <amorenoz@redhat.com>
1 parent 690e9c6 commit f6e2b2a

File tree

5 files changed

+74
-3
lines changed

5 files changed

+74
-3
lines changed

‎README.md‎

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -252,6 +252,37 @@ This workload is meant to fill with pause pods all the worker nodes from the clu
252252

253253
It creates two deployments, a client/curl and a server/nxing, and 1 service backed by the previous server pods. The client application has configured an startup probe that makes requests to the previous service every second with a timeout of 600s.
254254

255+
Note: This workload calculates the number of iterations to create from the number of nodes and desired pods per node. In order to keep the test scalable and performant, chunks of 1000 iterations will by broken into separate namespaces, using the config variable `iterationsPerNamespace`.
256+
257+
### node-density-cni with SRIOV VFs
258+
259+
It creates two deployments, a client/curl and a server/nxing, and 1 service backed by the previous server pods along with secondary network interfaces with SRIOV VFs. The client application has configured an startup probe that makes requests to the previous service every second with a timeout of 600s.
260+
261+
#### Pre-requisites for running with SRIOV VFs
262+
- Cluster should be installed with DAY-2 SRIOV Operator
263+
- If User want to run workload with Performance profile, It should be created.
264+
- Below SRIOV-network should be created before running the workload
265+
```yaml
266+
---
267+
apiVersion: sriovnetwork.openshift.io/v1
268+
kind: SriovNetwork
269+
metadata:
270+
name: sriov-net-node-density
271+
namespace: openshift-sriov-network-operator
272+
spec:
273+
ipam: |
274+
{
275+
"type": "whereabouts",
276+
"range": "172.20.0.0/16"
277+
}
278+
logLevel: info
279+
networkNamespace: node-density-cni-0
280+
resourceName: servervfs
281+
spoofChk: "off"
282+
trust: "on"
283+
```
284+
285+
255286
Note: This workload calculates the number of iterations to create from the number of nodes and desired pods per node. In order to keep the test scalable and performant, chunks of 1000 iterations will by broken into separate namespaces, using the config variable `iterationsPerNamespace`.
256287

257288
### node-density-heavy

‎cmd/config/node-density-cni/curl-deployment.yml‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,19 @@ spec:
88
labels:
99
name: curl-{{.Replica}}-{{.Iteration}}
1010
app: curl
11+
annotations:
12+
test: ovsperf
13+
{{if gt .numSriovs 0}}
14+
k8s.v1.cni.cncf.io/networks: '[{{range $i := untilStep 1 (int (add .numSriovs 1)) 1}}{{if ne $i 1}},{{end}}{"name":"{{$.sriovNetworkName}}","interface":"sriov{{$i}}"}{{end}}]'
15+
{{end}}
16+
{{if .perfProfile}}
17+
irq-load-balancing.crio.io: "disable"
18+
cpu-quota.crio.io: "disable"
19+
{{end}}
1120
spec:
21+
{{if .perfProfile}}
22+
runtimeClassName: performance-{{.perfProfile}}
23+
{{end}}
1224
topologySpreadConstraints:
1325
- maxSkew: 1
1426
topologyKey: kubernetes.io/hostname

‎cmd/config/node-density-cni/node-density-cni.yml‎

Lines changed: 11 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -46,7 +46,11 @@ metricsEndpoints:
4646
{{ end }}
4747

4848
jobs:
49+
{{ if .NUM_SRIOVS }}
50+
- name: node-density-cni-sriov
51+
{{else}}
4952
- name: node-density-cni
53+
{{ end }}
5054
namespace: node-density-cni
5155
jobIterations: {{.JOB_ITERATIONS}}
5256
qps: {{.QPS}}
@@ -67,12 +71,15 @@ jobs:
6771
pod-security.kubernetes.io/enforce: privileged
6872
pod-security.kubernetes.io/audit: privileged
6973
pod-security.kubernetes.io/warn: privileged
70-
objects:
7174

75+
objects:
7276
- objectTemplate: webserver-deployment.yml
7377
replicas: 1
7478
inputVars:
7579
nodeSelector: {{.NODE_SELECTOR}}
80+
perfProfile: {{.PERF_PROFILE}}
81+
numSriovs: {{.NUM_SRIOVS}}
82+
sriovNetworkName: {{.SRIOV_NETWORK_NAME}}
7683

7784
- objectTemplate: webserver-service.yml
7885
replicas: 1
@@ -81,3 +88,6 @@ jobs:
8188
replicas: 1
8289
inputVars:
8390
nodeSelector: {{.NODE_SELECTOR}}
91+
perfProfile: {{.PERF_PROFILE}}
92+
numSriovs: {{.NUM_SRIOVS}}
93+
sriovNetworkName: {{.SRIOV_NETWORK_NAME}}

‎cmd/config/node-density-cni/webserver-deployment.yml‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,19 @@ spec:
88
labels:
99
name: webserver-{{.Replica}}-{{.Iteration}}
1010
app: webserver
11+
annotations:
12+
test: ovsperf
13+
{{if gt .numSriovs 0}}
14+
k8s.v1.cni.cncf.io/networks: '[{{range $i := untilStep 1 (int (add .numSriovs 1)) 1}}{{if ne $i 1}},{{end}}{"name":"{{$.sriovNetworkName}}","interface":"sriov{{$i}}"}{{end}}]'
15+
{{end}}
16+
{{if .perfProfile}}
17+
irq-load-balancing.crio.io: "disable"
18+
cpu-quota.crio.io: "disable"
19+
{{end}}
1120
spec:
21+
{{if .perfProfile}}
22+
runtimeClassName: performance-{{.perfProfile}}
23+
{{end}}
1224
topologySpreadConstraints:
1325
- maxSkew: 1
1426
topologyKey: kubernetes.io/hostname

‎pkg/workloads/node-density.go‎

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -34,9 +34,9 @@ import (
3434
func NewNodeDensity(wh *workloads.WorkloadHelper, variant string) *cobra.Command {
3535
var rc int
3636
var metricsProfiles []string
37-
var iterationsPerNamespace, podsPerNode, churnCycles, churnPercent int
37+
var iterationsPerNamespace, podsPerNode, churnCycles, churnPercent, numSriovs int
3838
var podReadyThreshold, churnDuration, churnDelay, probesPeriod, pprofInterval time.Duration
39-
var containerImage, deletionStrategy, churnMode, selector string
39+
var containerImage, deletionStrategy, churnMode, selector, perfProfile, sriovNetworkName string
4040
var namespacedIterations, pprof, svcLatency bool
4141
var nodeSelector corev1.NodeSelector
4242
var matchExpressions []corev1.NodeSelectorRequirement
@@ -97,6 +97,9 @@ func NewNodeDensity(wh *workloads.WorkloadHelper, variant string) *cobra.Command
9797
AdditionalVars["PPROF"] = pprof
9898
AdditionalVars["PPROF_INTERVAL"] = pprofInterval.String()
9999
AdditionalVars["POD_READY_THRESHOLD"] = podReadyThreshold
100+
AdditionalVars["PERF_PROFILE"] = perfProfile
101+
AdditionalVars["NUM_SRIOVS"] = numSriovs
102+
AdditionalVars["SRIOV_NETWORK_NAME"] = sriovNetworkName
100103
nodeSelectorJson, err := json.Marshal(nodeSelector)
101104
if err != nil {
102105
log.Fatal(err.Error())
@@ -134,6 +137,9 @@ func NewNodeDensity(wh *workloads.WorkloadHelper, variant string) *cobra.Command
134137
case "node-density-cni":
135138
cmd.Flags().DurationVar(&podReadyThreshold, "pod-ready-threshold", 1*time.Minute, "Pod ready timeout threshold")
136139
cmd.Flags().BoolVar(&svcLatency, "service-latency", false, "Enable service latency measurement")
140+
cmd.Flags().IntVar(&numSriovs, "num-sriovs", 0, "Number of SR-IOV interfaces per pod")
141+
cmd.Flags().StringVar(&sriovNetworkName, "sriov-networkname", "sriov-net", "SR-IOV network name for IPAM configuration")
142+
cmd.Flags().StringVar(&perfProfile, "perf-profile", "", "Performance profile name in the Cluster")
137143
}
138144
cmd.Flags().StringSliceVar(&metricsProfiles, "metrics-profile", []string{"metrics.yml"}, "Comma separated list of metrics profiles to use")
139145
cmd.Flags().BoolVar(&namespacedIterations, "namespaced-iterations", true, "Namespaced iterations")

0 commit comments

Comments
 (0)