Skip to content

An simple example using parallelization function #28

Open
@caoxiaoyue

Description

Dear developers,

I notice nestle can support parallelization sampling now. can you give a simple example to show how to use it? I have made some attempt but failed. Below is my attempt.

from pathos.multiprocessing import ProcessingPool 
pool = ProcessingPool(nodes=4)  #use dill pickle

#below class is from nestle source code
class FakePool(object):
    """A fake Pool for serial function evaluations."""

    def __init__(self,pool):
        self.pool = pool

    def submit(self, fn, *args, **kwargs):
        return FakeFuture(fn, *args, **kwargs)

    def map(self, func, *iterables):
        return self.pool.map(func, *iterables)

    def shutdown(self):
        pass
    
class FakeFuture(object):
    """A fake Future to mimic function calls."""

    def __init__(self, fn, *args, **kwargs):
        self.fn = fn
        self.args = args
        self.kwargs = kwargs

    def result(self):
        return self.fn(*self.args, **self.kwargs)

    def cancel(self):
        return True

fakepool = FakePool(pool)


t1 = time.time()
res = nestle.sample(loglike, prior, 7, method='multi',pool=fakepool, queue_size=4,
                    npoints=25,callback=nestle.print_progress)
t2 = time.time()

Above method can run successfully, but the runing time of code do not decrease. Also, I type top command in linux shell, it seems above method do not spawn subprocess. could you give any suggestion?

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions