Skip to content

Conversation

@dkazanc
Copy link
Collaborator

@dkazanc dkazanc commented Dec 1, 2025

Fixes https://jira.diamond.ac.uk/browse/IMGDA-650

Before fixing the tests it would be great to discuss if this is the acceptable solution. It does, however, fix the problem mentioned in the issue.

The problem occurs when flats or darks are used with None or ignore flags in the YAML file. The reason for this is that parse_config in transform_loader_params1 builds the configuration files (DarksFlatsFileConfig) for D or F incorrectly. In that particular case, darks (wanted to be ignored) are created using the original dataset, basically darks = data. So I changed it in way that D or F are truly ignored when needed. So most of the keys are None, e.g.,

 darks_config = DarksFlatsFileConfig(
            file=input_file, data_path=None, image_key_path=None, ignore=True
        )

Another issue was around empty_shape[self.slicing_dim] = 0 making a zero array which cannot be flattened later in data reducer. So that np.mean(data, axis=axis, dtype=np.float32, out=reduced_data[0, :, :]) would result in NaNs in the output. And because we do not run data_checker on flats/darks after squashing, it destroys the result of the normalisation. Making it empty_shape[self.slicing_dim] = 1 resolves the problem.

Checklist

  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • I have made corresponding changes to the documentation

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants