tweak kontainer tests to reduce flake#17736
Conversation
| }); | ||
|
|
||
| // wait for driver to be active | ||
| driversPage.list().details(exampleDriver, 1).should('contain', 'Active'); |
There was a problem hiding this comment.
No check for Activating->Active status here because the driver may be updated to Active too quickly, introducing some flakiness
|
|
||
| driversPage.list().details(exampleDriver, 1).should('contain', 'Activating'); | ||
| driversPage.list().details(exampleDriver, 1).contains('Active', { timeout: 60000 }); | ||
| driversPage.list().details(exampleDriver, 1).should('contain', 'Active', LONG_TIMEOUT_OPT ); |
There was a problem hiding this comment.
Question
I think the LONG_TIMEOUT_OPT cannot be used on the .should.
That is for most of the cases here. Are you sure it is actually applying the timeout?
I faced this problem before. (And I think I still have similar codes not working)
I usually change to something like this.
driversPage.list().details(exampleDriver, 1).contains('Active', LONG_TIMEOUT_OPT);
I am happy to discuss if you are not sure. I did some tests but we can check together.
There was a problem hiding this comment.
You're right, thanks for catching this! I got .should('contain' and .contains(... mixed up at some point. I updated the PR to use .should('contain'... instead. Given that the tests pass anyway maybe these two timeouts are not needed at all? I'm kind of inclined to keep them just to be cautious
51f3560 to
a771e72
Compare
2fa6690 to
cc1d927
Compare
cc1d927 to
c169c7e
Compare
marcelofukumoto
left a comment
There was a problem hiding this comment.
- Removed Activating will reduce the Flakiness but reduce one test as well (ideally). It is usually short lived so I can imagine how that improve flakiness.
- Verifying the state on the UI additionally makes it less flaky as well.
- Added timeout.
LGTM
Summary
Fixes #17735
Occurred changes and/or fixed issues
This PR updates kontainer driver tests to remove a race condition. Instead of waiting for an Activating state in one driver before asserting an Activating state in another, which may have already progressed to Active, only the Active state is asserted in the bulk activation step.
The Activating -> Active status updates are still validated in the 'can create new driver' test
also in this pr:
Checklist
Admin,Standard UserandUser Base