Improve the way Python interpreters are selected in Preferences

[richlysakowski]

Problem Description

When I try to change to a different kernel in Spyder 4.2.5, either syntax for Windows or Linux gives error dialog: error: "Invalid file path".

Entering the path as follows gives the same error dialog: "Invalid file path"
c:\programdata\anaconda3\envs\webscrapers\python.exe

or

c:/programdata\anaconda3/envs/webscrapers/python.exe

What steps reproduce the problem?

1. Select the Tools... Preferences... Python Interpreter menu
2. Change the fully qualified interpreter path from one virtual environment to another (see above)
3. I get the spyder-kernels error message below

What is the expected output? What do you see instead?

I expect the new kernel path to be accepted and Spyder changes to the new kernel without error. Instead I get the error dialogs shown.

Versions

• Spyder version: 4.2.5
• Python version: 3.9.7
• Qt version: 5.12.9
• PyQt version: 5.12.3
• Operating System name/version: Windows 10

Dependencies

# Mandatory:
atomicwrites >=1.2.0            :  1.4.0 (OK)
chardet >=2.0.0                 :  4.0.0 (OK)
cloudpickle >=0.5.0             :  2.0.0 (OK)
diff_match_patch >=20181111     :  20200713 (OK)
intervaltree >=3.0.2            :  3.0.2 (OK)
IPython >=7.6.0                 :  7.30.1 (OK)
jedi =0.17.2                    :  0.17.2 (OK)
jsonschema >=3.2.0              :  4.3.2 (OK)
keyring >=17.0.0                :  23.4.0 (OK)
nbconvert >=4.0                 :  6.3.0 (OK)
numpydoc >=0.6.0                :  1.1.0 (OK)
paramiko >=2.4.0                :  2.9.1 (OK)
parso =0.7.0                    :  0.7.0 (OK)
pexpect >=4.4.0                 :  4.8.0 (OK)
pickleshare >=0.4               :  0.7.5 (OK)
psutil >=5.3                    :  5.8.0 (OK)
pygments >=2.0                  :  2.10.0 (OK)
pylint >=1.0                    :  2.12.2 (OK)
pyls >=0.36.2;<1.0.0            :  0.36.2 (OK)
pyls_black >=0.4.6              :  0.4.6 (OK)
pyls_spyder >=0.3.2             :  0.3.2 (OK)
qdarkstyle >=2.8;<3.0           :  2.8.1 (OK)
qtawesome >=0.5.7               :  1.1.1 (OK)
qtconsole >=5.0.3               :  5.2.2 (OK)
qtpy >=1.5.0                    :  2.0.0 (OK)
rtree >=0.8.3                   :  0.9.7 (OK)
setuptools >=39.0.0             :  60.2.0 (OK)
sphinx >=0.6.6                  :  4.3.2 (OK)
spyder_kernels >=1.10.2;<1.11.0 :  1.10.2 (OK)
textdistance >=4.2.0            :  4.2.2 (OK)
three_merge >=0.1.1             :  0.1.1 (OK)
watchdog >=0.10.3;<2.0.0        :  1.0.2 (OK)
zmq >=17                        :  22.3.0 (OK)

# Optional:
cython >=0.21                   :  None (NOK)
matplotlib >=2.0.0              :  3.5.1 (OK)
numpy >=1.7                     :  1.21.5 (OK)
pandas >=1.1.1                  :  1.3.5 (OK)
scipy >=0.17.0                  :  1.7.3 (OK)
sympy >=0.7.3                   :  None (NOK)

# Spyder plugins:
spyder_notebook                 :  0.3.2 (OK)
spyder_terminal                 :  0.3.1 (OK)

Sometimes if I restart Spyder after changing the kernel path to a different virtual environment kernel, it starts working with the newly selected kernel (prior to restarting). The dialog box does not tell the user to restart after changing the kernel. (We hope / expect that restarting will not be necessary. Other tools like PyCharm or VSCode do not require restart. )

[ccordoba12]

Hey @richlysakowski, thanks for reporting. Unfortunately, Spyder 4 is not maintained anymore. Could you check if our latest version (5.2.2) still has this problem?

You can use our Windows installer for that, which is very simple to download and install. You can find it here:

https://github.com/spyder-ide/spyder/releases

[richlysakowski]

Carlos,

Thank you for your response. I found out what the problem is !!!

I just checked with Spyder 5.1.5 and get the same GUI errors.

The problem is that I was selecting the wrong interpreter file. I was selecting "python.exe" instead of "pythonw.exe".

The result is the same whether I use the dialog box to select the interpreter or type it in manually:

C:\ProgramData\Anaconda3\envs\webscrapers\python.exe
C:/ProgramData/Anaconda3/envs/webscrapers/python.exe

Both path syntaxes (whether entered manually or selected using the GUI dialog) raise the same GUI error message, which is inaccurate.

I may have forgotten to mention that I am launching Spyder from a Windows shortcut. This should not matter because Spyder loads the Spyder its environment configuration (including interpreter location) from its own config file.

However, I was suspicious that I need a windowsless kernel for Spyder, since it provides its own GUI. When I check the Windows shortcut, it is configured to launch a "pythonw.exe" file, not python.exe" file.

The windows shortcut has this target string (I removed the CR-LF to make it easier to read):

C:\ProgramData\Anaconda3\pythonw.exe 
C:\ProgramData\Anaconda3\cwp.py 
C:\ProgramData\Anaconda3\envs\consuming_http 
C:\ProgramData\Anaconda3\envs\consuming_http\pythonw.exe 
C:\ProgramData\Anaconda3\envs\consuming_http\Scripts\spyder-script.py

When I select the pythonw.exe file, I do not get any error message.

C:/ProgramData/Anaconda3/envs/webscrapers/pythonw.exe

AFTER CHANGING THE KERNEL, THE CONSOLE KERNEL NEEDS TO BE RESTARTED TO PICK UP THE CHANGE:

The current iPython console does not pick up the kernel change, but when I restart the console's kernel, and the enter "import sys; sys.prefix" it shows me it is running the kernel that was just selected.

RECOMMENDED FIXES:
If your "ingenieros de sistemas" could please make the following fix and enhancement, it will prevent future users from experiencing the same confusing error.

1. update the dialog box to inform the user that they must select the "pythonw.exe" interpreter file, NOT the "python.exe" file.

2. In the "Preferences" GUI Panel, in the "Python Interpreter" tab, put in a sentence next to the file selection button that says something like "Navigate to your environment folder and select the file "pythonw.exe" (a windowless interpreter) and and not the "python.exe" file which runs in its own terminal window).

3. When the user selects the option "Use the following interpreter:", please
   highlight the button "Select File", so the user is visually prompted to select and press this button.

4. Please put a label ON or next to the "Select File" button so the button's purpose is obvious.

5. After the user selects the correct "pythonw.exe" file, then highlight the "Apply" button in the lower right hand corner.

Here are perhaps simpler ways to eliminate errors.

Make it so that the user cannot select the wrong file... apply a file-name and file-type filter in the Select File dialog box to display only the "pythonw.exe" file.

Or even simpler, have the user select the correct environment FOLDER and let Spyder find and select the "pythonw.exe" FILE.

#########################
Another future enhancement to make this entire process much easier and less error prone would be to have Spyder parse the Anaconda "envs" environments folder (e.g., "c:/programdata/anaconda3/envs") for the names of all available kernels and their paths and gives the user a list to chose from. This is what VSCode and PyCharm do now.

If I have more time later I will move these suggestions to Enhancements posts.

Here are my current versions and dependencies:

Mandatory:

atomicwrites >=1.2.0 : 1.4.0 (OK)
chardet >=2.0.0 : 4.0.0 (OK)
cloudpickle >=0.5.0 : 2.0.0 (OK)
cookiecutter >=1.6.0 : 1.7.3 (OK)
diff_match_patch >=20181111 : 20200713 (OK)
intervaltree >=3.0.2 : 3.0.2 (OK)
IPython >=7.6.0 : 7.29.0 (OK)
jedi >=0.17.2;<0.19.0 : 0.18.0 (OK)
jsonschema >=3.2.0 : 4.2.1 (OK)
keyring >=17.0.0 : 23.2.1 (OK)
nbconvert >=4.0 : 6.2.0 (OK)
numpydoc >=0.6.0 : 1.1.0 (OK)
paramiko >=2.4.0 : 2.8.0 (OK)
parso >=0.7.0;<0.9.0 : 0.8.2 (OK)
pexpect >=4.4.0 : 4.8.0 (OK)
pickleshare >=0.4 : 0.7.5 (OK)
psutil >=5.3 : 5.8.0 (OK)
pygments >=2.0 : 2.10.0 (OK)
pylint >=2.5.0;<2.10.0 : 2.7.2 (OK)
pyls_spyder >=0.4.0 : 0.4.0 (OK)
pylsp >=1.2.2;<1.3.0 : 1.2.4 (OK)
pylsp_black >=1.0.0 : None (OK)
qdarkstyle =3.0.2 : 3.0.2 (OK)
qstylizer >=0.1.10 : 0.2.1 (OK)
qtawesome >=1.0.2 : 1.1.0 (OK)
qtconsole >=5.1.0 : 5.1.1 (OK)
qtpy >=1.5.0 : 1.11.2 (OK)
rtree >=0.9.7 : 0.9.7 (OK)
setuptools >=49.6.0 : 58.2.0 (OK)
sphinx >=0.6.6 : 4.2.0 (OK)
spyder_kernels >=2.1.1;<2.2.0 : 2.1.3 (OK)
textdistance >=4.2.0 : 4.2.2 (OK)
three_merge >=0.1.1 : 0.1.1 (OK)
watchdog >=0.10.3 : 2.1.6 (OK)
zmq >=17 : 22.3.0 (OK)

Optional:

cython >=0.21 : None (OK)
matplotlib >=2.0.0 : 3.4.3 (OK)
numpy >=1.7 : 1.21.4 (OK)
pandas >=1.1.1 : 1.2.4 (OK)
scipy >=0.17.0 : 1.7.2 (OK)
sympy >=0.7.3 : None (OK)

[ccordoba12]

Hey @richlysakowski, thanks for your detailed answer. About your comments:

1. update the dialog box to inform the user that they must select the "pythonw.exe" interpreter file, NOT the "python.exe" file.

I think this probably has to do with the way you're launching Spyder because we have tests that check our validations for python.exe.

When the user selects the option "Use the following interpreter:", please
highlight the button "Select File", so the user is visually prompted to select and press this button.

Ok, we can do that.

Please put a label ON or next to the "Select File" button so the button's purpose is obvious.

I don't understand what you mean by this. Could you expand on it?

After the user selects the correct "pythonw.exe" file, then highlight the "Apply" button in the lower right hand corner.

When any change is done in our Preferences dialog, that button is changed from disabled to enabled. I think that's enough to let users know that they need to click it to apply their changes.

Or even simpler, have the user select the correct environment FOLDER and let Spyder find and select the "pythonw.exe" FILE.

Yeah, that would be a nice improvement and we have already thought about implementing something like that, but we haven't had time to do it.

Another future enhancement to make this entire process much easier and less error prone would be to have Spyder parse the Anaconda "envs" environments folder

We already do that, i.e. we automatically populate that option with all the conda and pyenv environments we can detect:

Again, our mechanism to detect them probably fails in your case due to the peculiar way you use to launch Spyder.