Skip to content

Commit 2236388

Browse files
committed
update the quickstart/faq
1 parent b948909 commit 2236388

File tree

2 files changed

+8
-4
lines changed

2 files changed

+8
-4
lines changed

docs/source/faq.rst

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -464,3 +464,8 @@ If those quick fixes don't work, feel free to raise an issue.
464464
However, as multi-threading and multi-processing are notoriously
465465
difficult to debug, especially on a problem I'm not familiar with,
466466
it's likely that I might not be able to help all that much.
467+
468+
469+
** How to decide on the number of processes in a pool and how to set queue_size**
470+
471+
Assuming that you decided on the number of live-points K that you want to use and that the likelihood evaluation is not very quick, you should use as many processes as you can up to around K. The queue_size should be equal the number of processes. If you are using the the number of processes that M is smaller than K, you may want to use :math:`M=K//2` or :math:`M=K//3` i.e integer fractions. So if you are using 1024 live-points all powers of two up to 1024 would be good choiceS for the number of processes.

docs/source/quickstart.rst

Lines changed: 3 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -412,6 +412,7 @@ argument::
412412
# initialize sampler with pool with pre-defined queue
413413
sampler = NestedSampler(loglike, ptform, ndim, pool=pool, queue_size=8)
414414

415+
There is *no* reason to set queue_size to anything other then the number of parallel processes in the pool.
415416
Parallel operations in `dynesty` are done by simply swapping in the
416417
`pool.map` function over the default `map` function when making likelihood
417418
calls. Note that this is a *synchronous* function call, which requires that
@@ -423,16 +424,14 @@ The reason why "parallel" is written in quotes above is that while function
423424
evaluations can be made in parallel, live point proposals must be done serially
424425
in order to avoid breaking the statistical properties of Nested Sampling.
425426
Assuming we are using :math:`M` processes with :math:`K` live points, this
426-
leads to sub-linear scaling :math:`S` of the form
427+
leads to sub-linear speed improvements :math:`S` of the form
427428
(`Handley et al. 2015 <https://arxiv.org/pdf/1506.00171.pdf>`_):
428429

429430
.. math::
430431
431432
S(M, K) = K \ln \left(1 + \frac{M}{K}\right)
432433
433-
This scales pretty linearly as long as the number of processes is much smaller
434-
than the number of live points, but falls off as the pool becomes relatively
435-
larger.
434+
This scales pretty linearly with the number of processes till the number of parallel processes is equal or larger than the number of live-points.
436435

437436
Depending on where the bottleneck of the computation lies, the provided
438437
`pool` can be disabled during certain function evaluations (e.g., when

0 commit comments

Comments
 (0)