FileNotFoundError after removing irrelevant files from working directory

[Psirving]

Hi,

I am running snakemake in the directory where I am developing the workflow, so files are being added/deleted as needed.

When using --dry-run or --executor slurm everything is groovy, but when I submit using --executor gbatch I get an error while building the DAG like the one below. In this case, I deleted the README, which is not involved in the workflow in any way. These errors are satisfied if I make an empty dummy file with this name. I'm not sure if this is related to the executor or something happening on gcloud. Have you seen this before, any ideas?

  File "/opt/conda/envs/snakemake/lib/python3.12/site-packages/snakemake/workflow.py", line 308, in get_files
    source_file_size = os.path.getsize(f)
                       ^^^^^^^^^^^^^^^^^^

  File "<frozen genericpath>", line 62, in getsize

FileNotFoundError: [Errno 2] No such file or directory: 'README.md'

[vsoch]

Is it possible something cached your directory state that needs to be reset / cleaned?

[Psirving]

I guess that's what I'm trying to figure out. I'm working in a vscode devcontainer. I've tried removing any files in the workspace that look like a cache and then rebuilding the container, but this error persists, so I don't think I am storing this info. It only happens when --executor gbatch is used. This made me wonder if the cache was on Google Cloud somewhere. I remove the snakemake-workflow-sources after each run, not sure if that is relevant.

[esebesty]

Did you ever figure out what to do with this? Getting the same error, after renaming a totally irrelevant shell script, that's not part of the Snakemake workflow.

I think snakemake is trying to archive all source files to a google bucket, based on code here that's referred in the error, and it fails if files are moved/renamed after an initial run. I guess everything is cached at the initial run, but no idea where to look for that cache.

[Psirving]

No, I never found a cache and didn't have the bandwidth to solve this. I'm pretty sure I would get this error again if I deleted the dummy file, but I haven't tested recently.

[esebesty]

Might have figured it out. Files are listed using self.dag.get_sources() found here. If these files are git managed, the are listed using git ls-files --recurse-submodules .. as shown here. Uncommitted changes will result in the old file name in the list, that was renamed. If I add/commit changes, the error goes away.

[Psirving]

Nice work! This feels right. I will test and report back later this week.

Submitting jobs on Google batch is part of my development/testing. This answer suggests that the plugin requires users to commit untested changes to Git repos. It would be nice to have a workaround. A comment near the code you linked suggests there is an unimplemented alternative.

# get git-managed files
# TODO allow a manifest file as alternative