Skip to content

Snakemake can't find retrieve_cutouts #1598

Open
@Eric-Nitschke

Description

@Eric-Nitschke

Checklist

  • I am using the current master branch or the latest release. Please indicate.
  • I am running on an up-to-date pypsa-eur environment. Update via conda env update -f envs/environment.yaml.

Describe the Bug

This bug might be OS specific, and might only occur on Windows.

When trying to run PyPSA-Eur for the first time, snakemake could not identify retrieve_cutouts as a rule which provides cutouts\europe-2013-sarah3-era5.nc. The issue arises for other cutouts as well, such as the tutorial one cutouts\be-03-2013-era5.nc.

Further investigations identified the following issue:

  • Through the changes introduced in PR Configurable cutout directory #1515, CDIR is now detrermined by CDIR = join(cutout_dir, ("" if run["shared_cutouts"] else RDIR)), instead of directly being set to "cutouts/".
  • join() is os.path.join().
  • at least on Windows join() adds a "\" after joining, resulting in "cutouts\". I assume that on Linux it adds "/", resulting in the old behavior.
  • I suspect that this leads to the behavior, where Snakemake thinks, that "cutouts\europe-2013-sarah3-era5.nc" can't be generated with retrieve_cutouts, which outputs CDIR + "{cutout}.nc". This might be, because "\e" and "\b" are escape characters.

Setting CDIR = "cutouts/" fixes the issue, but removes the improvements added by PR #1515. Maybe we could replace join() with something, that consistantly adds "/".

Error Message

Building DAG of jobs...
MissingInputException in rule determine_availability_matrix in file [...]\pypsa-eur\rules/build_electricity.smk, line 254:
Missing input files for rule determine_availability_matrix:
    output: resources/Test/[...]/availability_matrix_5_solar.nc
    wildcards: clusters=5, technology=solar
    affected files:
        cutouts\europe-2013-sarah3-era5.nc

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions