Improve cypress integration test performance and fix flaky tests#3350
Open
lresende wants to merge 16 commits intoelyra-ai:mainfrom
Open
Improve cypress integration test performance and fix flaky tests#3350lresende wants to merge 16 commits intoelyra-ai:mainfrom
lresende wants to merge 16 commits intoelyra-ai:mainfrom
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #3350 +/- ##
=======================================
Coverage ? 84.16%
=======================================
Files ? 157
Lines ? 19067
Branches ? 504
=======================================
Hits ? 16048
Misses ? 2827
Partials ? 192 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
eaf7544 to
0db71ba
Compare
0db71ba to
8fbd1e0
Compare
61f12da to
afbc593
Compare
d2d3ad7 to
6da2022
Compare
Replace the blanket return false that suppressed all uncaught exceptions with an allowlist of known benign JupyterLab errors. Unknown errors now propagate and fail the test, surfacing real bugs that were previously hidden. Signed-off-by: Luciano Resende <lresende@apple.com>
Replace cy.wait(1000) with a PUT request intercept guarded by a dirty-state check. When the document is dirty, wait for the server write to complete; when clean, skip the wait to avoid hanging. Confirm the document is no longer dirty afterward. Signed-off-by: Luciano Resende <lresende@apple.com>
Callers already close the tab after calling this command. The
internal closeTab caused "element not found" failures when the
tab was already gone. Also replaced synchronous content extraction
with a retryable should('contain.text') assertion.
Signed-off-by: Luciano Resende <lresende@apple.com>
Batch multiple file deletion patterns into a single find command instead of spawning N individual cy.exec() calls. This reduces setup/teardown overhead in tests that clean up many file types. Signed-off-by: Luciano Resende <lresende@apple.com>
- createPipeline: replace cy.wait(300) with
should('be.visible') on canvas drop div
- resetJupyterLab: add jp-Launcher visibility check so
callers don't need their own cy.wait after reset
Signed-off-by: Luciano Resende <lresende@apple.com>
- Replace cy.wait(300) with should('be.visible') on canvas
drop div for consistent pipeline editor readiness
- Guard runtime image select() calls with option existence
checks to prevent failures when SWR fetch hasn't resolved
- Extract selectRuntimeImage helper to reduce duplication
Signed-off-by: Luciano Resende <lresende@apple.com>
- Remove ~20 cy.wait() calls replaced with proper Cypress
retryable assertions and DOM readiness checks
- Sidebar switches now verified via .lm-mod-current selector
- createValidCodeSnippet waits for data-item-id to appear
- Editor readiness uses .elyra-ScriptEditor should('be.visible')
- saveAndCloseMetadataEditor no longer has a trailing wait
Signed-off-by: Luciano Resende <lresende@apple.com>
- Replace cy.wait(2000) with kernel idle status check (data-status="idle") before interacting with cells - Fix populateCells() stale DOM: re-query cells by index instead of using .each() with stale wrappers - Add openCellContextMenuWithSnippetItem() retry helper that dismisses and re-opens the menu up to 5 times until the extension command is registered - Use body click to dismiss menus instead of Esc key Signed-off-by: Luciano Resende <lresende@apple.com>
- pythoneditor: replace cy.wait(1000) for editor load with cm-content visibility check with timeout - pythoneditor: replace cy.wait(2000) for output with 15s timeout on output area assertion - pythoneditor: keep cy.wait(500) for LSP debounce with comment - submitnotebook: remove redundant cy.wait(2000) after reset since resetJupyterLab now waits for Launcher - submitnotebook: replace cy.wait(500) with notebook existence check after opening new notebook Signed-off-by: Luciano Resende <lresende@apple.com>
- Use deleteFiles batch command to reduce 12 individual cy.exec() calls to 2 batched find commands - Combine 3 elyra-metadata remove calls into a single shell command to reduce process spawn overhead Signed-off-by: Luciano Resende <lresende@apple.com>
- Bump defaultCommandTimeout from 8s to 10s to reduce spurious timeouts on slow CI runners - Lower numTestsKeptInMemory from 10 to 5 to reduce memory pressure during long test runs - Enable experimentalMemoryManagement for GC optimization - Switch cy:run to headless mode (~20% faster, no X11) - Add cy:debug script for local headed debugging Signed-off-by: Luciano Resende <lresende@apple.com>
6da2022 to
b8a9913
Compare
Signed-off-by: Luciano Resende <lresende@apple.com>
Signed-off-by: Luciano Resende <lresende@apple.com>
The openCellContextMenuWithSnippetItem helper asserted the context menu was 'visible', but in headless Electron the menu can render with 0x0 dimensions initially. The empty cell test does not need the retry helper since the snippet command is already registered (just disabled). Use a direct rightclick with a timeout on the specific menu item instead. Also change the retry helper's menu assertion from 'be.visible' to 'exist' for headless compatibility. Signed-off-by: Luciano Resende <lresende@apple.com>
In headless mode, the launcher takes longer to appear after closing the last tab. Add a launcher visibility check before clicking the launcher card to prevent cascading failures when the launcher hasn't rendered yet between tests. Signed-off-by: Luciano Resende <lresende@apple.com>
The extension command registers asynchronously even for empty cells, so the retry helper (dismiss and re-open menu) is needed here too. The previous fix incorrectly assumed the command was already registered for empty cells. Signed-off-by: Luciano Resende <lresende@apple.com>
7ba05c1 to
6cb8072
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
Improve cypress integration test performance and fix flaky tests
Fix flaky integration tests by replacing ~30 cy.wait() calls across all test files with proper Cypress retry-able assertions. Key changes:
Fixes #3301
How was this pull request tested?
Developer's Certificate of Origin 1.1