Set an overridable default Python

[dlukes]

I'd like to set a default Python version to use, but make it overridable via .pdm.toml. Any ideas how to achieve that?

What I've tried so far:

export PDM_PYTHON=/path/to/python

This does indeed set a default Python that PDM then uses, but it's not overridable via .pdm.toml. Trying to set a different version with pdm use prints WARNING: the config is shadowed by env var 'PDM_PYTHON', the value set won't take effect..

pdm config python.path /path/to/python

This creates a ~/.pdm/config.toml with the correct Python, but PDM ignores it in projects. Instead, if a local .pdm.toml doesn't exist in the project, PDM creates it as soon as you run a command which needs the config (e.g. pdm info) and sets python.path to the first Python suggested by pdm use (which is not necessarily the one I'd like to be the default, i.e. the one I set in ~/.pdm/config.toml). Not sure if this behavior is a bug or a feature?

In any case, any help appreciated!

[pawamoy]

Somewhat related to #831.
What about the other way around? The version in .pdm.toml is your default version, and you can temporarily change it with

PDM_PYTHON=/path/to/python pdm run command

[dlukes]

That probably works, but it doesn't really address the problem I'm having. I'll try and give more details (maybe I'm using PDM wrong): I have a project using PDM. I don't version control .pdm.toml, because the correct python.path can vary from computer to computer.

When I clone the repo on computer X, I'll typically run pdm sync to get started. But PDM then silently picks whatever Python it thinks is best, which might not be the Python I'm using as a daily driver and therefore intending to use with the project. (Specifically in my case, it picks Python 3.10 from Pyenv instead of 3.9 from Mambaforge.)

I already have computer X set up as I like it, with my dotfiles and all, so I could make sure PDM_PYTHON is set in my shell environment, so that the correct one gets used. But then pdm use stops working, because the env var always takes precedence, which makes experimenting with other Python versions than the default one cumbersome.

With that in mind, I guess I see two solutions (but please let me know if I'm just using PDM wrong and there's a totally different way to approach this):

1. Just set PDM_PYTHON, and when I need to use a different Python, either unset it and use pdm use or set it manually to something else. This is slightly inconvenient, but it works, requires no changes in PDM, and makes sure I don't accidentally use the wrong Python when cloning the repo to a new machine. Where it becomes really cumbersome is if I want to permanently use a different Python than the default one with a given project. Then it's the opposite footgun: if I don't remember to unset PDM_PYTHON every time, I'll end up incorrectly using the default Python. (Or I'd have to use additional tools which would autosource a .env file with the correct PDM_PYTHON whenever I cd into the project... This gets finicky and brittle fast.)
2. Or maybe PDM could stop guessing implicitly which Python to use, and instead, whenever there's no value set in .pdm.toml, it could prompt the user to decide -- as it does when running pdm init or pdm use. In that case, when I clone the repo and run pdm sync, PDM will ask me before setting python.path in .pdm.toml, it'll show me Python 3.10 first and Python 3.9 second, and I'll realize what's going on and pick the second (or whichever other one is appropriate in that specific case).

By the way, is there a situation in which the python.path setting in ~/.pdm/config.toml is honored? Because if not (and if it's intentional), then it would probably make sense to put a caveat in the docs, so that users know that there's no point in setting it.

[pawamoy]

I don't really see the problem in what you describe.

I don't track .pdm.toml either.
When I clone a repository, I select my Python interpreter by running pdm use python3.9 (or other versions) and that's it.

Since I'm testing my projects on multiple Python versions, I've written a small script that changes the selected version and runs the given command: https://github.com/pawamoy/copier-pdm/blob/master/project/scripts/multirun.sh. The script is called by my Makefile so I can just run make test to run pytest on every selected Python version: https://github.com/pawamoy/copier-pdm/blob/master/project/Makefile.jinja. At the end of the run, the Python interpreter in .pdm.toml is the latest I tested on. If I'm not happy with that, I just run pdm use -f python3.9 again.

[dlukes]

Thanks for this -- I'm starting to realize it might be primarily a mindset problem :) I've only just started using PDM after all. I need to get into the mindset that the workflow is git clone → pdm use -- that's how it's meant to work and that's what everyone does.

Still, I think what I've described under 2 in my previous reply would be a worthwhile enhancement. I.e., PDM would always prompt which Python to set, even when running pdm info or pdm sync, never implicitly picking the first available option. It would be a bit more ergonomic (one could avoid a separate pdm use on clone) and would avoid surprises in case one forgets, which is always helpful for newcomers.

Or do you think it's a bad fit for PDM?

[pawamoy]

If it prompts the user only when no Python interpreter is currently selected (i.e. no interperter is saved in .pdm.toml), why not 🙂
I'll let @frostming be the judge of that!

Also, I'll try enhancing my multirun script above so it knows how to automatically reactivate the previously selected version, allowing something like this:

• current: 3.9
• make test
  • old=3.9
  • use 3.7, test
  • use 3.8, test
  • use 3.9, test
  • use 3.10, test
  • use 3.11, test
  • use $old
• current: still 3.9!

[dlukes]

If it prompts the user only when no Python interpreter is currently selected (i.e. no interperter is saved in .pdm.toml), why not 🙂

Yep, that's what I meant :) I'm happy to open a separate issue if he thinks it would benefit from further discussion. (I'll wait for his opinion.)

I'll try enhancing my multirun script above so it knows how to automatically reactivate the previously selected version

👍