Skip to content

tweak kontainer tests to reduce flake#17736

Merged
mantis-toboggan-md merged 6 commits into
rancher:masterfrom
mantis-toboggan-md:17735-kontainer-flake
Jun 4, 2026
Merged

tweak kontainer tests to reduce flake#17736
mantis-toboggan-md merged 6 commits into
rancher:masterfrom
mantis-toboggan-md:17735-kontainer-flake

Conversation

@mantis-toboggan-md

@mantis-toboggan-md mantis-toboggan-md commented May 19, 2026

Copy link
Copy Markdown
Member

Summary

Fixes #17735

Occurred changes and/or fixed issues

This PR updates kontainer driver tests to remove a race condition. Instead of waiting for an Activating state in one driver before asserting an Activating state in another, which may have already progressed to Active, only the Active state is asserted in the bulk activation step.

The Activating -> Active status updates are still validated in the 'can create new driver' test

also in this pr:

  • fixed invalid timeout in 'create new driver' test
  • updated the delete driver test to use the action menu 'delete' option instead of re-testing the bulk action delete that is covered by another test
  • updated the assertion in the delete driver test to fail when the driver is still present
  • update activate driver test to ensure the example driver is actually inactive before the rest of the test runs
  • update the deactivate driver test to assert that the driver is actually reported as deactivated

Checklist

  • The PR is linked to an issue and the linked issue has a Milestone, or no issue is needed
  • The PR has a Milestone
  • The PR template has been filled out
  • The PR has been self reviewed
  • The PR has a reviewer assigned
  • The PR has automated tests or clear instructions for manual tests and the linked issue has appropriate QA labels, or tests are not needed
  • The PR has reviewed with UX and tested in light and dark mode, or there are no UX changes
  • The PR has been reviewed in terms of Accessibility
  • The PR has considered, and if applicable tested with, the three Global Roles Admin, Standard User and User Base

});

// wait for driver to be active
driversPage.list().details(exampleDriver, 1).should('contain', 'Active');

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No check for Activating->Active status here because the driver may be updated to Active too quickly, introducing some flakiness


driversPage.list().details(exampleDriver, 1).should('contain', 'Activating');
driversPage.list().details(exampleDriver, 1).contains('Active', { timeout: 60000 });
driversPage.list().details(exampleDriver, 1).should('contain', 'Active', LONG_TIMEOUT_OPT );

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Question
I think the LONG_TIMEOUT_OPT cannot be used on the .should.
That is for most of the cases here. Are you sure it is actually applying the timeout?
I faced this problem before. (And I think I still have similar codes not working)

I usually change to something like this.

    driversPage.list().details(exampleDriver, 1).contains('Active', LONG_TIMEOUT_OPT);

I am happy to discuss if you are not sure. I did some tests but we can check together.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right, thanks for catching this! I got .should('contain' and .contains(... mixed up at some point. I updated the PR to use .should('contain'... instead. Given that the tests pass anyway maybe these two timeouts are not needed at all? I'm kind of inclined to keep them just to be cautious

@nwmac nwmac force-pushed the 17735-kontainer-flake branch from cc1d927 to c169c7e Compare June 4, 2026 08:36

@marcelofukumoto marcelofukumoto left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Removed Activating will reduce the Flakiness but reduce one test as well (ideally). It is usually short lived so I can imagine how that improve flakiness.
  • Verifying the state on the UI additionally makes it less flaky as well.
  • Added timeout.

LGTM

@mantis-toboggan-md mantis-toboggan-md merged commit 6c7bb59 into rancher:master Jun 4, 2026
64 of 65 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

kontainer-drivers.spec.ts is flaky

2 participants