Skip to content

Commit b242cfd

Browse files
authored
Reduce parallelism to avoid OOMs in wheel tests (#5611)
Something appears to have changed between 23.10 and 23.12 causing the unit test suite to require more memory than before. As of now, I see no meaningful new commits on the 23.12 branch of cuml. The only real new commit is the addition of ARM CUDA 12 conda builds, which are completely independent. Therefore, the cause of the OOMs must be a change in a dependency. I do see multiple significant commits in raft, so perhaps one of those is the cause. NVIDIA/raft#1818 seems like the most plausible culprit, but that's coming from my position of absolute ignorance about raft and the fact that unintentionally copying data in mdspans would in principle be an easy way to accidentally increase memory usage. However, it really could be coming from anywhere. Authors: - Vyas Ramasubramani (https://github.com/vyasr) - Dante Gama Dessavre (https://github.com/dantegd) Approvers: - William Hicks (https://github.com/wphicks) - Jake Awe (https://github.com/AyodeAwe) URL: #5611
1 parent 9baf64b commit b242cfd

1 file changed

Lines changed: 1 addition & 1 deletion

File tree

ci/test_wheel.sh

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -22,5 +22,5 @@ python -m pip install $(echo ./dist/cuml*.whl)[test]
2222
if [[ "$(arch)" == "aarch64" && "${RAPIDS_BUILD_TYPE}" == "pull-request" ]]; then
2323
python ci/wheel_smoke_test_cuml.py
2424
else
25-
python -m pytest ./python/cuml/tests -k 'not test_sparse_pca_inputs' -n 8 --ignore=python/cuml/tests/dask && python -m pytest ./python/cuml/tests -k 'test_sparse_pca_inputs' && python -m pytest ./python/cuml/tests/dask
25+
python -m pytest ./python/cuml/tests -k 'not test_sparse_pca_inputs' -n 4 --ignore=python/cuml/tests/dask && python -m pytest ./python/cuml/tests -k 'test_sparse_pca_inputs' && python -m pytest ./python/cuml/tests/dask
2626
fi

0 commit comments

Comments
 (0)