Memory usage accumulates when script contains function definition

[Fanowitsch]

Description

What steps will reproduce the problem?

1. copy the script below into a new script (replace 1e8 with 1e7 if you have <32Gb of ram)
2. run while observing RAM usage (e.g. task manager)
3. re-run (without restarting the kernel)
4. repeatedly re-run -> RAM usage increases every time until kernel is restartet
5. the issue disappears when the (unused) function definition is omitted

def f(x):
    return x**2

A = list(range(int(1e8)))

ps.:

Versions

• Spyder version: 6.1.0 (conda)
• Python version: 3.12.7 64-bit
• Qt version: 6.9.1
• PyQt6 version: 6.9.1
• Operating System: Windows-11-10.0.26100-SP0

Dependencies

# Mandatory:
aiohttp >=3.11.2                                                  :  3.11.10 (OK)
asyncssh >=2.14.0,<3.0.0                                          :  2.21.1 (OK)
atomicwrites >=1.2.0                                              :  1.4.0 (OK)
bcrypt >=4.3.0                                                    :  5.0.0 (OK)
chardet >=2.0.0                                                   :  4.0.0 (OK)
cloudpickle >=0.5.0                                               :  3.0.0 (OK)
cookiecutter >=1.6.0                                              :  2.6.0 (OK)
diff_match_patch >=20181111                                       :  20200713 (OK)
github >=2.3.0                                                    :  2.8.1 (OK)
importlib_metadata >=4.6.0                                        :  7.0.1 (OK)
intervaltree >=3.0.2                                              :  3.1.0 (OK)
IPython >=8.15.0,<10.0.0,!=8.17.1,!=9.1.0,!=9.2.0,!=9.3.0,!=9.4.0 :  8.27.0 (OK)
ipython_pygments_lexers >=1.0                                     :  1.1.1 (OK)
jedi >=0.17.2,<0.20.0                                             :  0.19.1 (OK)
jellyfish >=0.7                                                   :  1.0.1 (OK)
jsonschema >=3.2.0                                                :  4.23.0 (OK)
keyring >=17.0.0                                                  :  24.3.1 (OK)
nbconvert >=4.0                                                   :  7.16.4 (OK)
numpydoc >=0.6.0                                                  :  1.7.0 (OK)
packaging >=20.0                                                  :  25.0 (OK)
parso >=0.7.0,<0.9.0                                              :  0.8.3 (OK)
pexpect >=4.4.0                                                   :  4.8.0 (OK)
pickleshare >=0.4                                                 :  0.7.5 (OK)
psutil >=5.3                                                      :  5.9.0 (OK)
pygments >=2.0                                                    :  2.15.1 (OK)
pylint >=3.1,<4                                                   :  3.3.8 (OK)
pylint_venv >=3.0.2                                               :  3.0.3 (OK)
pyls_spyder >=0.4.0                                               :  0.4.0 (OK)
pylsp >=1.13.0,<1.14.0                                            :  1.13.1 (OK)
pylsp_black >=2.0.0,<3.0.0                                        :  2.0.0 (OK)
pylsp_ruff >=2.3.0,<3.0.0                                         :  2.3.0 (OK)
pyuca >=1.2                                                       :  1.2 (OK)
qdarkstyle >=3.2.0,<3.3.0                                         :  3.2.3 (OK)
qstylizer >=0.2.2                                                 :  0.2.2 (OK)
qtawesome >=1.4.0,<1.5.0                                          :  1.4.0 (OK)
qtconsole >=5.7.0,<5.8.0                                          :  5.7.0 (OK)
qtpy >=2.4.0                                                      :  2.4.1 (OK)
rtree >=0.9.7                                                     :  1.0.1 (OK)
sphinx >=0.6.6                                                    :  7.3.7 (OK)
spyder_kernels >=3.1.0,<3.2.0                                     :  3.1.1 (OK)
superqt >=0.6.2,<1.0.0                                            :  0.7.6 (OK)
textdistance >=4.2.0                                              :  4.2.1 (OK)
three_merge >=0.1.1                                               :  0.1.1 (OK)
watchdog >=0.10.3                                                 :  4.0.1 (OK)
yarl >=1.9.4                                                      :  1.22.0 (OK)
zmq >=24.0.0                                                      :  25.1.2 (OK)

# Optional:
cython >=0.21                                                     :  None (NOK)
matplotlib >=3.0.0                                                :  3.9.2 (OK)
numpy >=1.7                                                       :  1.26.4 (OK)
pandas >=1.1.1                                                    :  2.2.2 (OK)
scipy >=0.17.0                                                    :  1.16.3 (OK)
sympy >=0.7.3                                                     :  1.13.2 (OK)

[ccordoba12]

Hey @Fanowitsch, thanks for reporting. I can confirm this behavior on Linux and it's very strange indeed. However, in my case Python frees the used memory after some time.

@dalthviz, what do you see on Windows?

[ccordoba12]

I also have a question for you @Fanowitsch: do you see the same behavior in a Jupyter notebook?

[dalthviz]

I was able to reproduce this on Windows. Checking, seems like a way to prevent this to happen (without having to remove the f function definition), is leaving checked the Run in console's namespace instead of an empty one option. So going to Run > Configuration per file and in the dialog going to the Custom configuration > Advanced settings section and checking there the mentioned option:

Could you check if setting that config in your setup improves the RAM usage @Fanowitsch ? Let us know!

[Fanowitsch]

I can confirm that this behaviour is absent in jupyternotebook.
I can confirm that this behaviour disappears when doing as @dalthviz suggested ("Run in console's namespace instead of an empty one" turned on)
I have not yet tried whether memory gets freed up after a while

Thanks a lot for the (incredibly!) quick help!