JOSE Review summary: package installation/environments

[r-ford]

Both JOSE reviewers had comments about how package installation and environments are discussed in Foundations. Since these comments are in a few locations, I'm using this issue to collect them, which will hopefully make the comments easier to address.

From #568:

If you were to include a requirements list, I think it would fit nicely in the Running Pythia Foundations examples section

From #574:

Installing and managing python with conda

I like the content on this page, but I expected a bit more of a mention of general concepts like virtual environments and dependency management, and conda as a tool for installing and managing them. There is a bit of this in the 'packages' section, but it pretty quickly goes to conda. I think exposing the reader to things like pip, venv, and newer tools like uv and pixi would be beneficial, even if text was included along the lines of 'in this material, we focus on conda environments but these are other types of environments/package installation/tools to be aware of...'

From openjournals/jose-reviews#308 (comment):

• Installation instructions: Is there a clearly stated list of dependencies?

  • Repo Installation

    • This is an online resource so I'm happy to waive missing dependencies list on the repo. There are some conda environment installation details as part of the contributor's guide for those wishing to install and develop locally.
    • However, I cannot find a clear list of developer dependencies in the docs. I was only able to see this by looking at the conda environment file. It is, of course, possible I missed this as the docs are extensive in which case my apologies.

  • Using/Examples:

    • Setup instructions suggest using anaconda or miniconda.

      • Given the changes to the anaconda licensing last year (see e.g. anaconda academia press release, datacamp or plain Python) I am glad that you promote miniconda and conda-forge here. (I also see the environment file is set up to use conda-forge.). However, the pros-cons do not discuss this point, instead focussing on installation size and GUI/IDE provision.
      • Whist this is an education resource so is covered by the license changes, it is intended to educate researchers and research activities are no longer covered. I feel this should be made clear.

  • Is there a reason there is no discussion of virtual environments and pip installation? This is, arguably, an equally if not more popular approach to installing python dependencies, and I'd expect researchers to encounter this in the wild.

    • I accept that it does not make sense to package the foundations project this way as installation would be much more challenging given external dependencies like NetCDF, ffmpeg, especially for newcomers to software and computing.
    • Often a smaller research package will not be available on conda-forge and instead needs to be installed from a local clone using pip, even if this is from within conda.

  • The expectation seems to be that users install the entire conda environment at the start and then work from this for all exercises.

    • In other contexts users should be creating their own environments/packaging for their projects and not rely on using the one from this resource which is discussed.
    • I would suggest you consider, for clarity, listing specific dependencies for each notebook/module at the start of that section to hammer home the concepts of dependencies and environments. This would also help those who want to work through a specific chapter or section rather than following from the very beginning.

[brian-rose]

This was discussed at today's Pythia Community meeting. The consensus was that we are well justified in sticking to an opinionated presentation of our recommended best practices for environment management, and we will continue to promote the use of mini forge and the Conda-forge channel.

We will update the text in Foundations to address:

• why we recommend against using pip (prevalence of dependencies on compiled libraries in geoscience workflows)
• existence of newer tools like pixi, with external links but no explicit recommendation
• Focus on use of the free, community-supported Conda-forge stack
• Mention existence of Anaconda as a commercially supported option for those that are interested

We will include instructions on the landing page for each book section for how to set up a minimal environment for that section alone, as part of our overall goal of teaching readers about package and environment management (but not require readers to do this). At the most granular notebook-by-notebook level, we will emphasize that import statements at the top of each notebook should give enough information for minimal environment descriptions.

[r-ford]

Note that a PR closing this issue also closes #568