modified call to multiprocessing.Pool;

now it creates min(len(policies), cpu_count) processes. This fixes #177 and replaces
#179. In general it does not hurt, previous behavior was to always create cpu_count()

processes, some were probably unused.

Author: enzbus

cvxportfolio/simulator.py

@@ -46,6 +46,7 @@
 import sys
 import time
 from itertools import starmap
+from os import cpu_count
 from pathlib import Path
 
 import numpy as np
@@ -770,7 +771,9 @@ def backtest_many(
         if (not parallel) or len(policies) == 1:
             result = list(starmap(self._worker, zip_args))
         else:
-            with Pool(initializer=_mp_init, initargs=(Lock(),)) as p:
+            with Pool(
+                    processes=min(len(policies), cpu_count()),
+                    initializer = _mp_init, initargs = (Lock(),)) as p:
                 result = p.starmap(self._worker, zip_args)
 
         return list(result)

docs/manual.rst

@@ -405,6 +405,7 @@ by providing a single objective and a single list of constraints, and specifying
 the ``planning_horizon`` argument to, for example, 2. This simply copies the terms
 for each step of planning horizon.
 
+.. _disk-access:
 
 Disk Access
 -----------
@@ -485,6 +486,34 @@ internet access. If you run the test suite (by ``python -m cvxportfolio.tests``)
 on a computer without internet access you should see a few tests, mostly
 in the ``test_data.py`` module, failing, but most of the test suite will run.
 
+Parallel back-testing
+---------------------
+
+You can run multiple back-tests in parallel wit the :meth:`MarketSimulator.backtest_many`
+method. It takes a list of policies and returns the corresponding list of
+:class:`cvxportfolio.result.BacktestResult`. Also
+:meth:`MarketSimulator.optimize_hyperparameters` uses the same approach,
+to search over the space of hyper-parameters efficiently.
+
+.. note::
+
+    It is not recommended to run multiple Cvxportfolio programs at the same time,
+    unless you are careful to not :ref:`access on-disk storage <disk-access>`. If
+    you want to run many back-tests at the same time, you should run a single program
+    with :meth:`MarketSimulator.backtest_many`.
+
+.. note::
+
+    If your Cvxportfolio program uses custom objects, for example
+    :doc:`a forecaster <examples/user_provided_forecasters>`,
+    and in that you call complex third party libraries, like machine-learning
+    ones, parallel back-testing can be problematic. You should in those cases
+    make sure to :ref:`initialize and finalize <execution-model>` all resources you use.
+    Cvxportfolio supports the ``multiprocess`` `parallel execution library
+    <https://multiprocess.readthedocs.io/en/latest/>`_, which may help in such
+    cases. Simply install ``multiprocess`` in the Python environment to make
+    Cvxportfolio use it.
+
 
 CVXPY
 -----
@@ -556,6 +585,8 @@ parallel back-tests, ....
 In the next section we explain how Cvxportfolio policy objects work,
 which is useful to know when extending them.
 
+.. _execution-model:
+
 Policy execution model
 ----------------------