env: add explicit cachetools dependency for Windows import error

[daniel-rdt]

Closes #748.

Changes proposed in this Pull Request

The PR proposes to add the cachetools package as an explicit dependency to avoid import errors on Windows installs as observed in the latest hands-on workshop.

Problem description

(AI summary)

Installing the open-tyndp Pixi environment on Windows can fail at runtime with ModuleNotFoundError: No module named 'cachetools'. The package is imported somewhere in the dependency stack but is not declared by any conda or PyPI package in the resolved graph, so the solver never guarantees its presence. On platforms where it happens to be pulled in transitively the import succeeds; on a clean Windows solve it can be absent, and the failure only surfaces at runtime.

Tasks

Workflow

Open issues

Notes

Checklist

• Security scan(s) show no high-severity bugs, critical vulnerabilities, or exposed secrets.
• I tested my contribution locally and it works as intended.
• Code and workflow changes are sufficiently documented.
• Changed dependencies are added to pixi.toml (using pixi add <dependency-name>).
• Changes in configuration options are added in config/config.default.yaml.
• Changes in configuration options are documented in doc/configtables/*.csv.
• Changes in configuration options are added in config/test/*.yaml.
• The multiple weather/climate years test is passing locally (using pixi run -e open-tyndp tyndp-cyears-test).
• Open-TYNDP SPDX license header added to all touched files.
• For new data sources or versions, these instructions have been followed.
• New rules are documented in the appropriate doc/*.rst files.
• A release note doc/release_notes.rst is added.
• Major features are documented with up-to-date information in doc/index.rst.
• Module docstrings added to new Python scripts.

[tgilon]

Thank you for the fix! I suggest adding this dependency to Windows only. Also, since the vulnerability scan has not yet been run on this PR, you cannot tick this item in the list. You also have to solve merge conflicts.

[daniel-rdt]

Also, since the vulnerability scan has not yet been run on this PR, you cannot tick this item in the list.

I thought the vulerability scan is automatically launched if required. Does one have to launch it manually?

[tgilon]

I thought the vulerability scan is automatically launched if required. Does one have to launch it manually?

I can't see it in the list of tests, which differ from #754 where the vulnerabilities are reported. I expect it to be resolved once #754 is merged into master.

[tgilon]

Let's wait for #754 before merging it

[daniel-rdt]

Let's wait for #754 before merging it

Agree. Thanks for adding the label @tgilon.

Another note on the lock files:

I investigated this morning that not all lock files were updated in this commit, despite the environment changes made.

Turns out this is expected: the pixi task sync-locks only exports lock files for the default environment which I believe we left on purpose. It seems like a natural addition to extend it to also cover the open-tyndp environment now that it is a dedicated pixi task. However, my tests show this currently fails due to a missing md5 hash in our vendored snakemake-minimal version.
I discussed this with @coroa and his suggestion is that the cleanest path is to drop our vendored snakemake dependency soon, since most of the underlying issues are now resolved upstream.

tl;dr: nothing to do right now, but let's keep in mind for later to:

• Remove our vendored snakemake dependency
• Update sync-locks to also export open-tyndp lock files

I opened a follow-up issue to track this.

[coroa]

Do we know what is importing cache-tools? I do you have an import stacktrace? This is something that an upstream dependency should add!

[daniel-rdt]

Do we know what is importing cache-tools? I do you have an import stacktrace? This is something that an upstream dependency should add!

In fact, cache-tools does not register as a tracable dependency in our stack. So I am not sure what caused the issue. We only realised this during a workshop on a participants machine. Installing it manually resolved the issue. Would you in that case prefer to leave it out? Or try to trace it back manually somehow? @coroa