chore(tests): add network perf tests for Retina

[ritwikranjan]

Description

This change adds network related performance tests for Retina. The test basically creates a new cluster in AKS, runs the tests defined in ritwikranjan/perf-tests, then installs retina in basic mode and runs the tests again. The results is then published after comparing the two results and obtaining the regressions. See below the sample results.
network-regression-results-20240922162603.json

Related Issue

If this pull request is related to any issue, please mention it here. Additionally, make sure that the issue is assigned to you before submitting this pull request.

Checklist

• I have read the contributing documentation.
• I signed and signed-off the commits (git commit -S -s ...). See this documentation on signing commits.
• I have correctly attributed the author(s) of the code.
• I have tested the changes locally.
• I have followed the project's style guidelines.
• I have updated the documentation, if necessary.
• I have added tests, if applicable.

Screenshots (if applicable) or Testing Completed

Please add any relevant screenshots or GIFs to showcase the changes made.

Additional Notes

Add any additional notes or context about the pull request here.

---

Please refer to the CONTRIBUTING.md file for more information on how to contribute to this project.

[SRodi]

Run a test on uksouth and getting this

                                --------------------------------------------------------------------------------
                                RESPONSE 400: 400 Bad Request
                                ERROR CODE: ErrCode_InsufficientVCPUQuota
                                --------------------------------------------------------------------------------
                                {
                                  "code": "ErrCode_InsufficientVCPUQuota",
                                  "details": null,
                                  "message": "Insufficient regional vcpu quota left for location uksouth. left regional vcpu quota 20, requested quota 36",
                                  "subcode": ""
                                }
                                --------------------------------------------------------------------------------
                Test:           TestPerfRetina

I also run the test in westus2, and that was not an issue, but I got the following:

2024/09/27 17:48:52 DaemonSet is not ready: kube-system/retina-agent. 0 out of 3 expected pods are ready
2024/09/27 17:48:54 DaemonSet is not ready: kube-system/retina-agent. 0 out of 3 expected pods are ready
2024/09/27 17:48:56 DaemonSet is not ready: kube-system/retina-agent. 0 out of 3 expected pods are ready
2024/09/27 17:48:58 DaemonSet is not ready: kube-system/retina-agent. 0 out of 3 expected pods are ready
2024/09/27 17:49:00 DaemonSet is not ready: kube-system/retina-agent. 0 out of 3 expected pods are ready
2024/09/27 17:49:02 Error received when checking status of resource retina-svc. Error: 'client rate limiter Wait returned an error: rate: Wait(n=1) would exceed context deadline', Resource details: 'Resource: "/v1, Resource=services", GroupVersionKind: "/v1, Kind=Service"
Name: "retina-svc", Namespace: "kube-system"'
2024/09/27 17:49:02 Retryable error? true
2024/09/27 17:49:02 Retrying as current number of retries 0 less than max number of retries 30
    runner.go:27: 
                Error Trace:    /home/srodi/src/retina/test/e2e/framework/types/runner.go:27
                                                        /home/srodi/src/retina/test/e2e/retina_perf_test.go:65
                Error:          Received unexpected error:
                                did not expect error from step InstallHelmChart but got error: failed to install chart: context deadline exceeded
                Test:           TestPerfRetina
DeleteResourceGroup setting stored value for parameter [SubscriptionID] set as [......-.....-....-....-.........]
DeleteResourceGroup setting stored value for parameter [ResourceGroupName] set as [srodi-e2e-netobs-1727452628]
DeleteResourceGroup setting stored value for parameter [Location] set as [westus2]
#################### DeleteResourceGroup ################################################################
2024/09/27 17:49:02 deleting resource group "srodi-e2e-netobs-1727452628"...
2024/09/27 17:49:05 resource group "srodi-e2e-netobs-1727452628" deleted successfully
--- FAIL: TestPerfRetina (3269.87s)

FYI @ritwikranjan

[timraymond]

What are we actually testing here? What assertions of behavior are being made?

[ritwikranjan]

This test basically generating regression data and not asserting anything.

[timraymond]

If that be the case, we should just create another binary artifact that runs these perf tests. The testing framework is just getting in the way since it wasn't designed for this purpose.

[SRodi]

@ritwikranjan I just got another fail on insufficient quota, this time for centralus. I would suggest to make sure the test can run in any regions part of locations slice. ([]string{"eastus2", "centralus", "southcentralus", "uksouth", "centralindia", "westus2"})

    runner.go:27: 
                Error Trace:    /home/srodi/src/retina/test/e2e/framework/types/runner.go:27
                                                        /home/srodi/src/retina/test/e2e/retina_perf_test.go:52
                Error:          Received unexpected error:
                                did not expect error from step CreateNPMCluster but got error: failed to finish the create cluster request: PUT https://management.azure.com/subscriptions/....-....-....-....-.........../resourceGroups/srodi-e2e-netobs-1727879517/providers/Microsoft.ContainerService/managedClusters/srodi-e2e-netobs-1727879517
                                --------------------------------------------------------------------------------
                                RESPONSE 400: 400 Bad Request
                                ERROR CODE: ErrCode_InsufficientVCPUQuota
                                --------------------------------------------------------------------------------
                                {
                                  "code": "ErrCode_InsufficientVCPUQuota",
                                  "details": null,
                                  "message": "Insufficient vcpu quota requested 32, remaining 0 for family standardDSv2Family for region centralus.",
                                  "subcode": ""
                                }
                                --------------------------------------------------------------------------------
                Test:           TestE2EPerfRetina
--- FAIL: TestE2EPerfRetina (26.22s)
FAIL
FAIL    command-line-arguments  26.239s
FAIL

[SRodi]

@ritwikranjan I am getting the following error while running the test based on the most recent commit

    runner.go:27: 
                Error Trace:    /home/srodi/src/retina/test/e2e/framework/types/runner.go:27
                                                        /home/srodi/src/retina/test/e2e/retina_perf_test.go:63
                Error:          Received unexpected error:
                                did not expect error from step GetNetworkPerformanceMeasures but got error: failed to get network performance measures: failed to execute tests: error getting CSV data from orchestrator pod: error reading logs from pod netperf-orch-59dsc: the server rejected our request for an unknown reason (get pods netperf-orch-59dsc)
                Test:           TestE2EPerfRetina

[ritwikranjan]

Will help with identifying issue #655