chore: Refactor sampler parallelism and add example-model workflows#39
chore: Refactor sampler parallelism and add example-model workflows#39
Conversation
KOVALW
left a comment
There was a problem hiding this comment.
I see no performance regression with these changes, the results are identifcal to those on wtk-mp, and the error message handling is helpful. I do think that this PR reveals 1) there are limits to reproducibility in the batching routine that are taken care of in the particle assignment routine, and 2) the Sampler class is doing heavy lifting that might require defining ParticleExecutor or Generation classes that contain some of the functionality. I'm comfortable merging these changes but want to flag these as issues
| results1 = sampler.run_parallel_batches( | ||
| max_workers=2, chunksize=2, batchsize=4 | ||
| ) | ||
| results2 = sampler.run_parallel_batches( |
There was a problem hiding this comment.
This test fails if batchsize is not the same. I would at least assert that chunksize and max_workers should be different so that we know the results are repeatable across different parallelization schemes in the case of shifting computational resources. For example, run_parallel and run_serial produce the same results regardless of max_workers specified in run_parallel.
This may suggest that we should be passing the spawned seed sequences for each desired accepted particle to the batch runner as well
KOVALW
left a comment
There was a problem hiding this comment.
I've added some small changes and this LGTM. I'll make an issue for reproducibility across batch processing particle requests to take advantage of the slot class
Summary
ABCSamplerinto smaller execution components for particle evaluation, progress/reporting, run-state tracking, and batch/particlewise generation.Changes
ParticleEvaluator,SamplerRunState,SamplerReporter, typed sampler request/result carriers, and dedicated generation runners.python -m example_model, createsbenchmarks/automatically, and ignores generated benchmark artifacts.conftest.pypath setup and expanded tests around repeatability, runner behavior, worker defaults, output suppression, and error cleanup.