Skip to content

Can cpnest be made to accept a pool of processes? #41

@cdcapano

Description

@cdcapano

The fact that CPNest currently tries to handle parallel processing on its own presents it from being able to use MPI (as noted in Issue #33), and also makes it a pain to work with. Most (all?) other samplers that I've countered allow a preallocated pool of processes to be passed to them, which gives the user greater freedom to control the parallel environment in the manner that best fits their environment. For example, in PyCBC Inference, we create a pool of processes ourselves (which allows us to side step pickling issues when using python's multiprocessing), then give that pool to the sampler. As a result, we immediately can use either multiprocessing or MPI for parallelization for all other samplers.

If CPNest used a pool instead of trying to manage parallelization on its own, then it would make it easier both on our end to use it, and on the CPNest developers' end to support parallelization, since now you would not need to add code to deal with different types of communication schemes, negating issues like #33.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions