Running too many processes/threads and overloading the CPUs

[albrechtsenmm]

Hi Miles,

Thanks for providing such a well documented library and for the active support. This is truly helping the greater scientific community.

I have been trying to run my model on our HPC, but the jobs get killed due to using to many threads. I have provided the model parameters below.

model = PySRRegressor( niterations=100000, ncycles_per_iteration=2000, binary_operators=["+", "-", "*", "/", "pow"], unary_operators=["exp", "log"], model_selection="best", elementwise_loss="L1DistLoss()", maxsize=35, populations=120, weight_optimize=0.001, turbo=True, parsimony=0.01, adaptive_parsimony_scaling=1000, parallelism="multiprocessing", procs=40, progress=False, verbosity=0, output_jax_format=False, temp_equation_file=False, run_id=run_id )

I have attempted to limit the number of threads to 1 per core, but it seems to be overwritten somehow.
export JULIA_NUM_THREADS=1 export OMP_NUM_THREADS=1 export MKL_NUM_THREADS=1 export OPENBLAS_NUM_THREADS=1

Using these parameters, the julia code spawns 44 cores and 147 threads. I wonder if the population or turbo parameter might spawn more procs than requested?

[MilesCranmer]

Hi @albrechtsenmm,

Thanks for the question. Sorry for the confusion: The correct environment variable is PYTHON_JULIACALL_THREADS, as PySR interfaces with Julia using juliacall. You can see the full list of env vars here: https://juliapy.github.io/PythonCall.jl/stable/juliacall/#julia-config.

It would certainly be nice if we had these listed in the PySR docs as well.

This threads parameter is set at startup, so the PySRRegressor config isn’t able to unset it unfortunately.

Let me know if that env var works for you?