[9.4] [Security Solution] Fix flaky Cypress tests that use preventPrebuiltRulesPackageInstallation helper (#271457)#271920
Merged
Conversation
…RulesPackageInstallation` helper (elastic#271457) Closes elastic#268359, closes elastic#268360, closes elastic#268548, closes elastic#268549, closes elastic#268566, closes elastic#268567, closes elastic#268708, closes elastic#269232, closes elastic#269233, closes elastic#269723, closes elastic#270225, closes elastic#270226, closes elastic#270318, closes elastic#270319, closes elastic#270611, closes elastic#271178, closes elastic#271179, closes elastic#271182, closes elastic#271183, closes elastic#271193, closes elastic#271543, closes elastic#271544. This PR fixes flakiness in Cypress tests that use `preventPrebuiltRulesPackageInstallation` helper. 🟢 Flaky Test Runner: https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/12493 Test used to fail with `Error: Socket closed before finished writing response`. We are using `cy.intercept(...)` to intercept initialization flow requests to skip package installation. However, if initialization request also contains another flow that's unrelated to package installation we don't mock the request. If the non-mocked request is in flight, and during this time Cypress reloads a page (`cy.reload()`), the request gets canceled and Cypress treats it as an error, although it's a normal situation. The fix is using a listener instead of `request.continue(...)`. Listener doesn't error if test runner navigates away – it just doesn't run if response wasn't received. The flakiness was introduced in this PR: elastic#266690 However, the flaky test runner didn't catch it back then because I didn't run all the specs that were affected by the change in helper 🤦♂️. I updated my skill to make sure it triggers the Flaky Test Runner for tests that use a shared helper. (cherry picked from commit 3bb21b1)
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
Contributor
Author
💛 Build succeeded, but was flaky
Failed CI StepsTest Failures
Metrics [docs]
cc @nikitaindik |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Backport
This will backport the following commits from
mainto9.4:preventPrebuiltRulesPackageInstallationhelper (#271457)Questions ?
Please refer to the Backport tool documentation