Memory allocation problem when running 127 jobs on joplin

[sandrinebedard]

When running 127 jobs in parallel on joplin for preprocessing, this error occurs in gradunwarp for some subjects:

MemoryError: Unable to allocate 208. KiB for an array with shape (208, 256) and data type float32

gradunwarp uses a lot of memory as discussed here

Solution:
I trided reducing the number of jobs running in parallel with sct_run_batch -> with the flag -jobs 120, everything went smoothly. I would conitnue this way for future preprocessing of the rest of the Uk biobank dataset. Not sure how to add this to the README; maybe put a warning or something like that because this solution is for this case specifically.

[jcohenadad]

Right, I also ran into some problem when maxing out the available cores. For the spine-generic pipeline, i used maximum 20 cores (!) to avoid running into problems with the sct_deepseg_sc function, which maxes out the RAM.

[sandrinebedard]

Ok thanks for the info! I will remember that for when I generate the segmentations! On Compute Canada, we are creating batches of 32, should we be worried that the RAM will be maxed out also? (I will discover it quickly either way)

[jcohenadad]

32 on compute canada should be fine (i ran aggressive pipelines with that number of cores, so it should be OK)