Conversation
TorreZuk
left a comment
There was a problem hiding this comment.
New tests have been added, and can reduce after variability of runs is reduced.
|
Please add some details on how many more tests were added and how much longer the execution time needs. Eventually, this can be captured automatically with prolense. CC @amd-shiraz |
The newest tests are for SYTRS and SYTRS2; there are 3456 new test cases. I was able to run these tests in about 47 seconds on gfx942. But because of the test variance on TheRock CI, we also saw individual test cases taking multiple seconds in the nightly tests. |
| # 24 mins + 20% margin = 30 mins => ~40 mins (considering gpu delays and lags) | ||
| "timeout_minutes": 60, | ||
| # Extended tests on math-ci take approx 5 hrs (as of May 5, 2026) | ||
| "timeout_minutes": 120, |
There was a problem hiding this comment.
let's work on test standardization for rocsolver soon! https://github.com/ROCm/TheRock/blob/main/docs/development/test_filtering.md
having a developer wait 360 minutes for tests to run in a PR is not good
perhaps, could we do just standard tests instead of extended? and if nightly, we run extended?
Motivation
Nightly rocSOLVER tests intermittently time out on theRock CI. The variance in test time will need to be investigated, but increasing the time out value was suggested as a short-term solution.
Technical Details
Increased test time out for rocSOLVER from 60 mins to 120 mins.
Submission Checklist