Feature serialized stepsampler#167
Open
JohannesBuchner wants to merge 9 commits into
Open
Conversation
Old behaviour: When running with MPI, step samplers can progress at different efficiencies. The point of the first one that finishes N steps is accepted and this increases the nested sampling likelihood threshold. The other step samplers are informed and may revert / back track to an earlier state. This can cause some areas of the parameter space to be explored much slower, and thus become undersampled, leading to biases. see #159 New behaviour: Reverting is disabled. The step sampler remembers the Lmin it was started with, and returns the Lmin->L edge to the ReactiveNestedSampling integrator, which stores it and may use it (unless the new Lmin is already above L). This should reduce the issue, but is still not perfect, as the edge may arrive too late. Ideally, we should wait until all stepsamplers at the current Lmin have finished, but this is difficult to synchronize. Resuming could pick up the late edges; however, we would need to sort the point store by Lmin. This should be technically possible within an improvement loop if we keep track of the last pointstore entry at the start of the loop, and keep the entries since then sorted.
Without reverting, a stepsampler may return a result, but it cannot be used anymore, even when resuming it would probably not be loaded. With reverting, raising the Lmin should affect only the stepsamper with probability 1/K, so the sampler has a good chance of not having to revert back very often. With the new queue, variation in executation duration should be less of an issue.
Owner
Author
|
It would be great if someone could test this further with MPI for problematic behaviour, and which part of the diff introduces it. For example, what happens if _create_point and pointpile is left to the old code? |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
addresses issue #159 , please help testing!