Issue around absent either darks or flats #656
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes https://jira.diamond.ac.uk/browse/IMGDA-650
Before fixing the tests it would be great to discuss if this is the acceptable solution. It does, however, fix the problem mentioned in the issue.
The problem occurs when flats or darks are used with
Noneorignoreflags in the YAML file. The reason for this is thatparse_configintransform_loader_params1builds the configuration files (DarksFlatsFileConfig) for D or F incorrectly. In that particular case, darks (wanted to be ignored) are created using the original dataset, basicallydarks = data. So I changed it in way that D or F are truly ignored when needed. So most of the keys are None, e.g.,Another issue was around
empty_shape[self.slicing_dim] = 0making a zero array which cannot be flattened later in data reducer. So thatnp.mean(data, axis=axis, dtype=np.float32, out=reduced_data[0, :, :])would result in NaNs in the output. And because we do not run data_checker on flats/darks after squashing, it destroys the result of the normalisation. Making itempty_shape[self.slicing_dim] = 1resolves the problem.Checklist