Incompatible snakemake versions between different environments lead to issues

[koen-vg]

Checklist

• I am using the current master branch or the latest release. Please indicate.
• I am running on an up-to-date pypsa-eur environment. Update via conda env update -f envs/environment.yaml.

Describe the Bug

When I was just starting a fresh pypsa-eur repo, I was getting the below error during retrieval rules:

AttributeError: 'Snakemake' object has no attribute '_params_store'

After just a little bit of digging, it looks like snakemake has changed how it internally refers to parameters, and my take is that the environment.yaml and retrieve.yaml have different version requirements for snakemake, leading to two different versions, one of which stored parameters in one way, and one in the other way.

The suggested fix is to somehow ensure that exactly the same snakemake version is used in both environments.

[lkstrp]

Ref #1469

Hot fix is to simply not use retrieve.yaml and only use environment.yaml/ pinned envs.

The retrieve.yaml was not really maintained and I dropped it in 1469. I think you were the one who added it in the first place. What is the use case for it? I am mainly curious. We can have multiple environments, but they should all have a pinned version and need to be checked/ maintained via the CI. At the moment this is only done for the base environment.yaml.

[koen-vg]

Yes so if anything this shows that #1469 is well-timed :)

If I remember correctly, one of my main motivations was that I thought it was annoying that development changes in environment.yaml would cause snakemake to trigger reruns for all retrieval rules, since the software environment changed. While we, the developer, might know that bumping the pandas version or whatever doesn't have anything to do with whether the data bundle needs to be retrieved again or not, snakemake doesn't know that.

Granted, there are of course other ways of fixing this, like some sort of snakemake [...] --touch run, or blanket use of --rerun-triggers mtime. If retrieve.yaml causes trouble or is a pain to maintain, then the occasional snakemake --touch run might be worth it.

Hope this makes some sense.