Description
Which tests failed?
*** note ***
transferred from kgateway back to gloo because it's not relevant in kgateway
copied over the issue description, but not all the comments. see kgateway-dev#10250 for comment history
Which tests failed?
TestDiscoveryWatchlabels/Discovery/TestDiscoverySpecPreserved
Logs:
=== RUN TestDiscoveryWatchlabels/Discovery/TestDiscoverySpecPreserved
service/example-svc created
=== NAME TestDiscoveryWatchlabels
status.go:71:
Timed out after 20.001s.
The function passed to Eventually failed at /home/runner/work/gloo/gloo/test/kubernetes/testutils/assertions/status.go:70 with:
have matcher for namespace discovery-watchlabels-test which is not found
service "example-svc" deleted
Initial Investigation
This has been reproducible locally, though infrequently, whereas it seems to occur much more frequently in CI
This flake occurs when an Upstream created by Discovery does not have its status populated
The failure occurs here, on the assertion that eventually an Upstream, which we expect Discovery to create, has an "Accepted" namespaced status
When debugging I found that occasionally the Upstream is created and never gets a status, which leads that assertion to fail
I'm not sure what would cause an Upstream not to get a status, but it may be notable that this failure seems to exclusively occur on the second test, TestDiscoverySpecPreserved, and not TestDiscoverUpstreamMatchingWatchLabels, suggesting that there may be some form of test pollution
Additional Information
One example of a failure on nightlies
┆Issue is synchronized with this Asana task by Unito
Initial Investigation
No response
Additional Information
No response