Issue around absent either darks or flats

[dkazanc]

Fixes https://jira.diamond.ac.uk/browse/IMGDA-650

Before fixing the tests it would be great to discuss if this is the acceptable solution. It does, however, fix the problem mentioned in the issue.

The problem occurs when flats or darks are used with None or ignore flags in the YAML file. The reason for this is that parse_config in transform_loader_params1 builds the configuration files (DarksFlatsFileConfig) for D or F incorrectly. In that particular case, darks (wanted to be ignored) are created using the original dataset, basically darks = data. So I changed it in way that D or F are truly ignored when needed. So most of the keys are None, e.g.,

 darks_config = DarksFlatsFileConfig(
            file=input_file, data_path=None, image_key_path=None, ignore=True
        )

Another issue was around empty_shape[self.slicing_dim] = 0 making a zero array which cannot be flattened later in data reducer. So that np.mean(data, axis=axis, dtype=np.float32, out=reduced_data[0, :, :]) would result in NaNs in the output. And because we do not run data_checker on flats/darks after squashing, it destroys the result of the normalisation. Making it empty_shape[self.slicing_dim] = 1 resolves the problem.

Checklist

• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes
• I have made corresponding changes to the documentation

[dkazanc]

@yousefmoazzam I guess we can close this one. This bug with the flattening of the array is probably not relevant as you re-wrote how dummy arrays are generated?

[yousefmoazzam]

With the changes I did in #683, the bug regarding the array not being flattened could still happen, just that it now doesn't happen with the particular darks/flats case that was originally being dealt with.

I made a quick comment a few weeks ago as to when the bug could happen, see #683 (comment).

We can fix it of course, but it's probably better to do in a separate PR, since this one has changes with the darks/flats stuff that aren't relevant after the refactoring in #683.